Python for MATLAB Development
Python for MATLAB Development
Extend MATLAB with 300,000+ Modules from the Python Package Index
Albert Danial
Table of Contents
Chapter 1: Introduction 1
1.1 Learn Python Through MATLAB Equivalents . 2
1.2 Is Python Really Free? . 4
1.3 What About Toolboxes? . 5
1.4 Why Python Won’t Replace MATLAB . 5
1.5 Contents at a Glance . 6
1.6 I Already Know Python. How Do I Call Python Functions in MATLAB? . 8
1.7 The Recipes Don’t Work! MATLAB Crashes! (and What to Do About It) . 8
Chapter 2: Installation 11
2.1 Downloads 11
2.1.1 Match Your Python and MATLAB Versions! . 12
2.1.2 Verify That Python Runs 12
2.2 Post-Install Configuration and Checkout . 12
2.3 Creating and Running a Python Program 14
2.4 The Curse of Choice 15
2.5 Virtual Environments . 15
2.5.1 matpy, the Virtual Environment Used in This Book . 16
2.5.2 Commands to Manage Virtual Environments 17
2.5.3 Keeping Your Virtual Environment Current 18
About the Author xxi
About the Technical Reviewers xxiii
Preface xxv
Acknowledgments xxviivi
2.6 ipython, IDEs . 20
2.6.1 Autoload Modules When ipython Starts 20
2.7 Python and MATLAB Versions Used in This Book 21
Chapter 3: Language Basics 23
3.1 Assignment . 23
3.1.1 Assignment with = . 23
3.1.2 In-Place Updates with +=, -=, and Others . 25
3.1.3 Walrus Operator, = 26
3.2 Printing 27
3.3 Indentation 28
3.3.1 Tabs 30
3.4 Indexing 30
3.4.1 Brackets vs. Parentheses . 30
3.4.2 Zero-Based Indexing and Index Ranges . 30
3.4.3 Start, End, and Negative Indices . 31
3.4.4 Index Strides . 32
3.4.5 Index Chaining 32
3.5 for loops 33
3.5.1 Early Loop Exits 36
3.5.2 Exit from Nested Loops 36
3.6 while Loops . 37
3.7 if Statements . 38
3.7.1 Boolean Expressions and Operators . 39
3.7.2 Range Tests 39
3.8 Functions 40
3.8.1 Pass by Value and Pass by Reference 41
3.8.2 Variable Arguments 43
3.8.3 Keyword Arguments . 44
Table of ConTenTsvii
3.8.4 Decorators 46
3.8.5 Type Annotation and Argument Validation 47
3.8.6 Left-Hand Side Argument Count . 49
3.9 Generators 50
3.9.1 yield, next() . 50
3.9.2 range() 51
3.10 Scoping Rules and Global Variables 52
3.11 Comments . 54
3.11.1 Docstrings 54
3.12 Line Continuation 56
3.13 Exceptions . 56
3.14 Modules and Packages . 58
3.14.1 Namespace . 58
3.14.2 def main() . 60
3.14.3 Module Search Path . 61
3.14.4 Installing New Modules 62
3.14.5 Module Dependency Conflicts and Virtual Environments . 63
Chapter 4: Data Containers 65
4.1 NumPy Arrays 66
4.2 Strings . 68
4.2.1 Strings, Character Arrays, and Byte Arrays . 68
4.2.2 String Operations 70
4.2.3 Formatting 74
4.2.4 Separate a String into Words 75
4.2.5 Tests on Strings 76
4.2.6 String Searching, Replacing with Regular Expressions 77
4.2.7 String Templates . 80
4.3 Python Lists and MATLAB Cell Arrays . 81
4.3.1 Initialize an Empty List . 82
4.3.2 Create a List with Given Values 83
Table of ConTenTsviii
4.3.3 Get the Length of a List 83
4.3.4 Index a List Item . 84
4.3.5 Extract a Range of Items 85
4.3.6 Warning—Python Index Ranges Are Not Checked! 87
4.3.7 Append an Item 90
4.3.8 Append Another List . 91
4.3.9 Preallocate an Empty List . 92
4.3.10 Insert to the Beginning (or Any Other Position) of a List . 93
4.3.11 Indexing Nested Containers 93
4.3.12 Membership Test: Does an Item Exist in a List? . 94
4.3.13 Find the Index of an Item 95
4.3.14 Apply an Operation to All Items (List Comprehension) 97
4.3.15 Select a Subset of Items Based on a Condition 98
4.3.16 How Many Times Does an Item Occur? 99
4.3.17 Remove the First or Last (or Any Intermediate) List Item . 99
4.3.18 Remove an Item by Value . 100
4.3.19 Merging Multiple Lists 101
4.3.20 Unmerging Combined Lists 103
4.3.21 Sort a List . 105
4.3.22 Reverse a List . 106
4.4 Python Tuples 106
4.5 Python Sets and MATLAB Set Operations . 109
4.6 Python Dictionaries and MATLAB Maps 111
4.6.1 Iterating over Keys 112
4.6.2 Testing for Key Existence 113
4.6.3 Iterating over Keys, Sorting by Key . 115
4.6.4 Iterating over Keys, Sorting by Value 117
4.6.5 Tuples As Keys 119
4.6.6 List Values 120
Table of ConTenTsix
4.7 Structured Data . 120
4.7.1 Method 1: namedtuple 121
4.7.2 Method 2: SimpleNamespace . 121
4.7.3 Method 3: Classes 122
4.7.4 Method 4: Data Classes 124
4.7.5 Enumerations . 132
4.8 Caveat: “=” Copies a Reference for Nonscalars! 134
Chapter 5: Dates and Times 137
5.1 Time 137
5.1.1 Current Time . 137
5.1.2 Time String Formats . 138
5.1.3 tic, toc; %timeit . 140
5.2 Dates . 141
5.2.1 datetime Objects to and from Strings . 141
5.2.2 Time Deltas . 142
5.3 Timezones . 144
5.3.1 UTC vs. Local Time . 146
5.4 Time Conversions to and from datetime Objects 148
5.4.1 Unix Epoch Seconds . 148
5.4.2 ISO 8601 Time String 149
5.4.3 Julian Date; Modified Julian Date; GPS Time . 150
5.5 zoneinfo in Python >= 3.9 151
5.5.1 List Available Timezones . 152
5.5.2 Date Increments Across Daylight Savings Transition 152
5.6 References 153
Chapter 6: Call Python Functions from MATLAB 155
6.1 Configure MATLAB to Recognize Python . 155
6.2 Does It Work? 157
6.3 Importing (and Reloading) Python Modules 159
6.4 Configure startup.m for Python Work 159
Table of ConTenTsx
6.5 Create Python Variables and Call Python Functions in MATLAB 161
6.5.1 Scalars 161
6.5.2 Lists and Cell Arrays . 163
6.5.3 Tuples . 165
6.5.4 Numeric Arrays . 166
6.5.5 Dictionaries and Structs . 168
6.5.6 Keyword Arguments . 171
6.5.7 Python-to-MATLAB and MATLAB-to-Python Variable Converters . 172
6.5.8 Traversing Generators . 172
6.5.9 Traversing zip() . 174
6.6 Modifying the Python Search Path Within MATLAB . 174
6.6.1 Extending sys.path with an Alias 175
6.6.2 Extending sys.path with insert() . 175
6.6.3 Extending sys.path with append() 176
6.7 Python Bridge Modules . 177
6.8 Debugging Python Code Called by MATLAB 178
6.9 Summary of Steps to Calling Python Code from MATLAB . 181
6.10 Call MATLAB from Python . 182
6.10.1 Install matlab.engine 182
6.10.2 Call Functions in a New MATLAB Session 183
6.10.3 Call Functions in an Existing MATLAB Session . 184
6.11 Other Mechanisms for MATLAB/Python Interaction 186
6.11.1 System Calls and File I/O 186
6.11.2 TCP/IP Exchange . 188
Chapter 7: Input and Output 189
7.1 Text Files . 189
7.1.1 Reading Corrupted Text Files with pathlib 191
7.1.2 Reading and Writing Numeric Data 192
7.1.3 I/O Exceptions . 192
7.1.4 Parsing Text 193
Table of ConTenTsxi
7.1.5 csv 199
7.1.6 XML . 203
7.1.7 YAML . 212
7.1.8 JSON . 213
7.1.9 ini 214
7.2 Recipe 7-1: Read YAML Files . 216
7.3 Recipe 7-2: Write YAML Files 217
7.4 Recipe 7-3: Read an ini File 218
7.5 Recipe 7-4: Write an ini File 220
7.6 Binary Files . 222
7.7 Excel .xls, .xlsx 225
7.7.1 Reading .xls and .xlsx Files 226
7.7.2 Writing .xlsx Files . 231
7.8 Recipe 7-5: Write an .xlsx File . 234
7.9 HDF5 . 237
7.9.1 Reading an HDF5 File . 237
7.9.2 Writing an HDF5 File . 239
7.9.3 Reading and Writing HDF5 Dataset Attributes 243
7.9.4 Iterating over All HDF5 Datasets . 244
7.10 NetCDF4 246
7.10.1 Reading a NetCDF4 File 246
7.11 SQLite 249
7.12 Recipe 7-6: CRUD with an SQLite Database 252
7.13 Pickle Files 255
7.14 MATLAB .mat Files 257
7.14.1 Inspecting the Contents of a .mat File 258
7.14.2 Reading a .mat File . 260
7.14.3 Writing a .mat File 262
7.14.4 mat Version 7.3 . 267
Table of ConTenTsxii
7.15 Command-Line Input 268
7.15.1 Python: sys.argv . 269
7.15.2 MATLAB: Function Arguments; varargin . 270
7.15.3 Python: argparse 271
7.16 Interactive Input 275
7.17 Receiving and Sending over a Network 276
7.17.1 HTTP, HTTPS 276
7.17.2 Python As a Web Server . 278
7.17.3 TCP/IP . 280
7.18 Recipe 7-7: TCP Server . 284
7.19 Interacting with Databases . 286
7.19.1 PostgreSQL . 286
7.19.2 MongoDB 290
7.20 Recipe 7-8: CRUD with a PostgreSQL Database . 293
7.21 Recipe 7-9: CRUD with a MongoDB Database . 296
7.21.1 Read . 298
7.21.2 Update 300
7.21.3 Delete . 300
7.22 Recipe 7-10: Interact with Redis . 300
7.23 Reference 305
Chapter 8: Interacting with the File System 307
8.1 Reading Directory Contents 308
8.2 Finding Files 309
8.3 Deleting Files 309
8.4 Creating Directories 310
8.5 Deleting Directories 310
8.6 Walking Directory Trees 311
Table of ConTenTsxiii
Chapter 9: Interacting with the Operating System and External Executables 315
9.1 Reading, Setting Environment Variables . 315
9.2 Calling External Executables 317
9.2.1 Checking for Failures 319
9.2.2 A Bytes-Like Object Is Required . 321
9.3 Inspecting the Process Table and Process Resources 322
Chapter 10: Object-Oriented Programming 327
10.1 Classes 327
10.1.1 Private vs. Public 329
10.1.2 Custom Printers 330
10.1.3 Custom Exceptions . 331
10.2 Performance Implications . 333
Chapter 11: NumPy and SciPy 335
11.1 NumPy Arrays 335
11.1.1 Formatting NumPy Array Values . 336
11.1.2 Differences Between NumPy Arrays and MATLAB Matrices . 338
11.1.3 NumPy Data Types 341
11.1.4 Typecasting Scalars and Arrays 343
11.1.5 Hex, Binary, and Decimal Representations . 344
11.1.6 Creating Arrays . 345
11.1.7 Complex Scalars and Arrays . 355
11.1.8 Linear Indexing . 357
11.1.9 Reading/Writing Arrays to/from Text Files 359
11.1.10 Reading/Writing Arrays to/from Binary Files 360
11.1.11 Primitive Array Operations 368
11.1.12 Adding Dimensions . 371
11.1.13 Array Broadcasting . 373
11.1.14 Index Masks . 377
11.1.15 Extracting and Updating Submatrices 380
Table of ConTenTsxiv
11.1.16 Finding Terms of Interest 383
11.1.17 Object-Oriented Programming and Computational Performance . 388
11.2 Linear Algebra . 394
11.2.1 Linear Equations . 395
11.2.2 Singular Value Decomposition 397
11.2.3 Eigenvalue Problems 398
11.3 Sparse Matrices 401
11.3.1 Sparse Matrix Creation with COO, CSC, CSR . 401
11.3.2 Sparse Matrix Creation with LIL, DOK . 408
11.3.3 Sparse Matrix Creation with BSR, DIA 411
11.3.4 Test Matrices 413
11.3.5 Sparse Matrix I/O 415
11.3.6 Linear Algebra 418
11.3.7 Summary of Sparse Formats and Capabilities; Recommendations . 424
11.4 Interpolation 426
11.4.1 One-Dimensional Interpolation . 426
11.4.2 Two-Dimensional Interpolation . 429
11.4.3 Two-Dimensional Interpolation on a Grid 435
11.5 Curve Fitting 437
11.5.1 Linear Regression . 437
11.5.2 Fitting Higher-Order Polynomials . 438
11.5.3 Fitting to Models . 439
11.6 Recipe 11-1: Curve Fitting with differential_evolution() . 447
11.7 Regression 449
11.7.1 Ordinary Least Squares 449
11.7.2 Weighted Least Squares . 452
11.7.3 Confidence and Prediction Intervals . 453
11.8 Recipe 11-2: Weighted Least Squares in MATLAB 457
11.9 Recipe 11-3: Confidence and Prediction Intervals in MATLAB 460
Table of ConTenTsxv
11.10 Finding Roots 463
11.10.1 Univariate . 463
11.10.2 Multivariate . 465
11.11 Recipe 11-4: Solving Simultaneous Nonlinear Equations . 466
11.12 Optimization 467
11.12.1 Linear Programming . 468
11.12.2 Simulated Annealing . 474
11.13 Differential Equations 479
11.14 Symbolic Mathematics 481
11.14.1 Defining Symbolic Variables . 482
11.14.2 Derivatives 482
11.14.3 Integrals . 483
11.14.4 Solving Equations . 484
11.14.5 Linear Algebra 484
11.14.6 Series . 486
11.15 Recipe 11-5: Using SymPy in MATLAB 486
11.16 Recipe 11-6: Compute Laplace Transforms 487
11.17 Unit Systems . 488
11.17.1 Defining Units in pint 489
11.18 Recipe 11-7: Using pint in MATLAB . 491
11.19 References 492
Chapter 12: Plotting 493
12.1 Point and Line Plots 493
12.1.1 Saving Plots to Files . 496
12.1.2 Multiple Plots per Figure 497
12.1.3 Date and Time on the X Axis . 499
12.1.4 Double Y Axes . 500
12.1.5 Histograms . 502
12.1.6 Stack Plots 504
Table of ConTenTsxvi
12.2 Area Plots 505
12.2.1 imshow() . 505
12.3 Animations 510
12.4 Plotting on Maps with Cartopy 514
12.4.1 Points 514
12.4.2 Lines . 517
12.4.3 Area 519
12.4.4 MATLAB and Cartopy 527
12.4.5 Avoid matplotlib’s Qt Backend in MATLAB! . 529
12.5 Recipe 12-1: Drawing Lines on Maps with Cartopy 529
12.6 Recipe 12-2: Overlay Contours on Globe with Cartopy . 531
12.7 Recipe 12-3: Shade Map Regions by Value with Cartopy . 532
12.8 Plotting on Maps with GeoPandas 535
12.9 Making Plots in Batch Mode . 535
12.10 Interactive Plot Editing 536
Chapter 13: Tables and Dataframes 539
13.1 Loading Tables from Files . 540
13.2 Table Summaries 543
13.2.1 Table Size, Column Names, Column Types 543
13.2.2 summary() and .info()/.describe() . 545
13.2.3 groupsummary() and .value_counts() . 546
13.2.4 head() and tail() 546
13.3 Cleaning Data 547
13.3.1 Renaming Columns 547
13.3.2 Changing Column Data Types . 548
13.3.3 Changing Column Data . 549
13.3.4 Making Timestamps from Strings 550
13.4 Creating Tables Programmatically 550
13.5 Sorting Rows . 551
Table of ConTenTsxvii
13.6 Table Subsets 552
13.6.1 All Rows, Selected Columns . 552
13.6.2 All Columns, Selected Rows . 553
13.6.3 Selected Rows, Selected Columns . 553
13.6.4 Filter Rows by Conditional Operations 555
13.7 Iterating over Rows . 555
13.8 Pivot Tables . 556
13.8.1 Single-Level Aggregation . 556
13.8.2 Multilevel Aggregation 558
13.9 Adding Columns 558
13.10 Deleting Columns 560
13.11 Joins Across Tables . 561
13.12 GeoPandas 564
13.13 Recipe 13-1: Maps with GeoPandas . 569
13.14 References 573
Chapter 14: High Performance Computing 575
14.1 Paths to Faster Python Code . 576
14.2 Reference Problems 577
14.2.1 The Mandelbrot Set 577
14.2.2 A 2D Finite Element Solver . 579
14.3 Reference Hardware and OS . 581
14.4 Baseline Performance . 582
14.4.1 Mandelbrot Set Performance . 582
14.4.2 FE Solver Performance . 582
14.5 Profiling Python Code 583
14.5.1 Scalene . 584
14.5.2 Austin and FlameGraph 586
14.6 Multicore Computation with multiprocessing . 589
14.7 Vectorization . 591
Table of ConTenTsxviii
14.8 Cython . 594
14.8.1 Python Compiled with Cython . 595
14.8.2 Parallel for Loops with Cython 598
14.8.3 Cython Performance . 599
14.9 Pythran 599
14.9.1 Examples of Signature Comments . 600
14.9.2 Python Compiled with Pythran; Parallel for Loops 601
14.9.3 Pythran Performance 603
14.10 Numba . 604
14.10.1 Parallel for Loops with Numba . 606
14.10.2 Numba Keyword Arguments nopython, fastmath . 607
14.10.3 Numba Performance 607
14.10.4 Numba Limitations . 607
14.11 f2py . 608
14.12 Recipe 14-1: Accelerating MATLAB with Python on a Single Computer 612
14.12.1 Compile Python Modules in a MATLAB-Friendly Virtual Environment 612
14.12.2 MATLAB + Cython . 612
14.12.3 MATLAB + Pythran 613
14.12.4 MATLAB + Numba 613
14.12.5 MATLAB + f2py . 614
14.12.6 MATLAB + Python Performance Results . 614
14.13 Distributed Memory Parallel Processing with Dask 615
14.13.1 Parallel MATLAB . 616
14.13.2 Dask Execution Paradigm and Performance Expectations . 617
14.13.3 Example 1: Sum of Prime Factors on One Computer 618
14.13.4 Setting Up a Dask Cluster on Multiple Computers 625
14.13.5 Example 2: Sum of Prime Factors on Multiple Computers . 626
14.13.6 Example 3: A Gigapixel Mandelbrot Image . 628
14.13.7 Example 4: Finite Element Frequency Domain Response . 634
Table of ConTen
Index
A
Altair, 493
Anaconda, 203
Animations
ball bouncing, 511–513
key elements, 510, 511
matplotlib, 510
Application program interface (API), 276
arange() function, 52
Area plots
imagesc(), 506
imshow(), 506, 507
bilinear interpolation, 507, 508
prefixes, 508, 509
astimezone() method, 150
Austin, 586–589
B
Basic linear algebra subroutines
(BLAS), 602
Block compressed row (BSR), 411
Bokeh, 493
Bridge modules, 177
built-in scope, 52
C
Cartopy
cartographic transformation, 514
drawing lines, map, 529, 530
MATLAB, 527, 528
overlay contours, globe, 531, 532
projection functions, 514
shade map regions, value, 532–534
Cell arrays, 163, 165
Class
circle, 327
constructor method, 329
custom exceptions, 331, 333
custom printers, 330
init() function, 328
instances, 329
private vs. public, 329, 330
variables, 327
Cloud service, 644
Coiled’s cloud
performance, 644, 645
running, 644
Command-line input
argparse, 271, 273
function arguments, 270
sys module, 269, 270
Comma-separated value (CSV), 199, 539
Comments
block comment, 54
definition, 54
docstrings, 54
Communication-intensive parallel
program, 576
Compressed Sparse Column (CSC), 406
Compressed Sparse Row (CSR), 406
Computer Algebra System (CAS), 481
conda package manager, 62
Conda environment, 612
conda-meta directory, 19
conda update, 12
Converters, 172
create_dataset() function, 239, 240
Create, read, update, delete (CRUD), 252,
254, 255
Curve fitting
differential_evolution()
bridge module, 447
compute_ABCD() function, 448
cost function, 447
matlab.engine, 447
py module, 447
solution, 449
fit models
curve_fit(), 442–444
differential_evolution(), 446
differential_evolution(), 446
r2_score() function, 442
scipy.optimize module, 440
scipy.optimize.curve_fit(), 440
test data, 441
linear regression, 437
polynomials, 438, 439
Cython, 594
MATLAB, 612
parallel for loop, 598
performance, 599
Python, 595, 597, 598
D
Dask, 615
execution paradigm, 617, 618, 620
Mandelbrot set, 628, 633
multiple computer, 625–627
parallel MATLAB, 616
prime factor, 623
scheduler, 622, 639
single computer, 625
Data
cleaning, 547
columns
changing, 549, 550
data types, 548
renaming, 547
timestamps, 550
Data classes
convenience mechanism, 124
dynamic modification, 128, 129
entity relationship, 127
field values, 125, 126
calling mag(), 125
MATLAB vs. Python, 124
point, 124
traversing, 130
type enforcement, 124
type validation, Pydantic, 131, 132
Data containers, 6
MATLAB, 65
nonscalars, 134, 135
Python, 65
Dataframes., see Tables
Dates
datetime, 141
string, 141
time deltas, 142, 143
datestr() function, 138
datetime functions, 147
datetime module, 141
datetime object
definition, 148
ISO 8601 time string, 149, 150
Julian date, 150, 151
Unix epoch seconds, 148, 149
INDEX689
Debugging
Python code
diary mode, 180, 181
do_work(), 179, 180
functions, 178, 179
logging, 178
steps, 181, 182
trace module, 178
deepcopy() function, 658
Dictionaries
add keys, 170, 171
creation, 170
datasets, 111
iterating over keys
for key in dict, 112
sorting by key, 115–117
sorting by values, 117–119
key existence
.get() method, 113, 114
.get() method, 114
key collision, 115
KeyError, 113
.setdefault() method, 113, 114
test_key, 113
limitation, 170
vs. lists, 112
Python
country-to-capital city
relationship, 112
subscript, 112
struct, 169
tuples, 119
values, 120
Dictionary of keys (DOK), 408
Differential equations, 479, 480
Direct frequency, 652, 654
Directories
contents, 308
creation, 310
deletion, 310
expressions, 308
trees
.csv files, 311
dir(), 311
find() functions, 313
generators, 313
os.walk(), 311
output, 312
printing, 312
traversing, 311
.xlsx files, 311, 313
Duffing oscillator equation, 479
E
eig() eigenvalue function, 49
enclosing scope, 52
enumerate() function, 35
Enumerations, 132, 133
env command, 14
Excel .xls, .xlsx
definition, 225
extract columns, 229
extract rectangular block, 230, 231
extract rows, 228, 229
get worksheet names, 227
reading files, 226, 227
writing, 231, 233
writing file, 233–236
Exceptions, 56, 57
executemany() function, 252
External executables
bytes-Like Object, 321
CalledProcessError, 319
ffmpeg, 319
os.environ object, 317
INDEX690
Result, 318
Result.returncode, 319
Status, 318
subprocess module, 317
subprocess.CalledProcessError, 321
subprocess.run(), 317, 319, 321
subsystem.run(), 319
system() function, 317
F
f2py command, 608, 610, 614
File system
deletion, 309, 310
finding, 309
MATLAB, 307, 308
Python, 307, 308
Finite element (FE), 579, 634
FlameGraphs, 587–589
For loops
definition, 33
examples, 34
exits, 36
nested loops, 36, 37
variables, 34
Fortran-compiled module, 611
Functions
comparison operators, 39
decorators, 46
keyword arguments, 44, 45
left-hand side argument count, 49
lists/dicts/arrays, 42, 43
pass by values/pass by
reference, 41
type annotation/argument
validation, 47, 48
variable arguments, 43
G
Generators
definition, 50
range(), 51, 52
yield, next(), 50
geo* functions, 514
GeoPandas
create plot, 566
dataframes, 565
data type, 566
geographic data, 564
join, 566
maps
bridge functions, 571, 572
bridge module, 569
issues, 573
iterrows(), 570, 571
plotting, 535
read_file() function, 565
shapefile, 565
Zillow research, 564
zip code, 567, 568
global keyword, 52
global scope, 52
H
h5dump command-line utility, 241
HDF5
datasets, 244
netcdf4, 246, 248
reading files, 237, 239
reading/writing dataset attributes, 243
writing files, 239–243
Hierarchical Data File (HDF), 237
High performance computing
(HPC), 7, 575
External executables (cont.)
INDEX691
baseline performance
FE, 582, 583
Mandelbrot set, 582
multiprocessing, 589
profiling tools
austin, 586–589
drawbacks, 584
flameGraphs, 587–589
scalene, 584, 586
reference hardware/OS, 581
reference problems, 577
FE, 579, 580
Mandelbrot set, 577
I
If statements
Boolean expressions/operators, 39
range tests, 39
variables, 38
imshow() function, 505
Indentation
brackets/parentheses, 30
index chaining, 32
indexing methods, 30
index strides, 32
MATLAB and Python, 29
start/end/negative indices, 31
Tabs, 30
zero-based indexing/index ranges, 30
Index masks
creation, 378, 379
mathematical operations, 377, 378
uses, 379
input() function, 275
Installation, MATLAB
creating/running Python program, 14
downloads, 11, 12
package, 11
post-install configuration/
checkout, 12, 13
toolboxes, 15
Interpolation
one-dimensional
extrapolation, 426
linear, 426, 427
spline, 427, 428
two-dimensional
cubic/linear methods, 434
evaluation, 432, 433
function, 429
grid, 435–437
griddata(), 430
meshgrid(), 430
np.meshgrid(), 431, 432
testing, 429
values, 429
Ints, 223
ipython, IDEs
application navigator, 20
autoload modules, 20
version, 21
iterables, 41
iterrows(), 53
J
JavaScript Object Notation (JSON)
complex data structures, 187
MATLAB to Python, 187
Python to MATLAB, 187, 188
Just-in-time (JIT), 604, 657
K
Keyword arguments, 171, 172
INDEX692
L
Lambda function, 116
Linear algebra
eigenvalue problems, 398–400
linear equations, 395, 396
MATLAB vs.Python, 394, 395
SVD, 397, 398
Linear indexing, 357–359
Linear programming (LP) problems
integer-valued, 469
linprog(), 468
bridge module, 472
coefficients, 469
MATLAB, 471, 472
Python solution, 470
relationships, 469
sol object, 470
solution status, 470
objective function, 468
PuLP module, 473
server properties, 468
Linux, 316
list() function, 194
List of lists (LIL), 408
Lists, 163, 165
local scope, 52
Local variables, then enclosing, global,
and built-in (LEGB), 52
Long expressions, 56
M
macOS, 316
Mandelbrot set, 577
MATLAB .mat files
data containers, 257–260
definition, 257
reading files, 260, 261
v7.3, 267, 268
writing files, 262, 264–267
MATLAB
Anaconda’s Python distribution, 157
call functions
existing session, 184, 185
new session, 183
cell arrays, 81
configuration
Linux/macOS, 157
matlab command, 155
path, 156
pyenv command, 156
Windows, 156, 157
container.Map, 168
definition, 1
eigensolvers, 398, 399
environment variables, 158
errors, 158
functions, 3, 4
horzcat() function, 91
hybrid environment, 157
incompatible library function, 157
install engine, 182
install module, 183
libraries, 8
lower-level libraries, 158
maps, 111
matpy, 8, 9
py module, 1
Python, 2, 6
pyversion(), 183
software, 9
software engineering, 157
toolboxes, 5
MATLAB-native variable, 650
MATLAB performance, 612
MATLAB/Python interaction
INDEX693
file I/O, 186
system calls, 186
TCP/IP exchange, 188
matplotlib module, 9, 493
Matrix slices, 656
maxsplit keyword, 194
MB() function, 606, 648, 651
Mixed-type mathematical
expressions, 655
Modules/packages
definition, 58
def main(), 60
dependency conflicts/virtual
environments, 63
installation, 62
namespace, 58, 59
search path, 61
MongoDB, 290–292
bridge module, 296
delete, 300
read, 298, 299
updates, 300
mset function, 301
multiprocessing module, 589
N
Natural Earth project, 523, 564
ncinfo command-line tool, 246
Network communication protocols
HTTP, HTTPS, 276, 277
MATLAB client, 283, 284
Python as a web server, 278
TCP/IP, 280–283
TCP server, 284, 286
Numba, 604
keyword argument, 607
limitations, 607, 608
Mandelbrot program, 605
MATLAB, 613
parallel for loop, 606
performance, 607
Python, 604
Numeric arrays
constants, 167
conversion, 166
MATLAB
commands, 167
functions, 167
single-precision floats, 166
type, 166
NumPy
definition, 335
eigensolvers, 398, 399
NumPy and SciPy, 7
NumPy arrays
decimal to binary, 344
hexadecimal to decimal, 345
MATLAB, 66
scalars, 339
operations
addition/subtraction, 368
elementwise, 369
Hermitian, 370
transpose, 370
terms of interest
find() function, 388
smallest term, 385
sorted data, 387
text files, reading, 359
typecasting, 343
NumPy arrays
binary files
endian conversions, 366, 368
metadata, 362, 363
reading, 361, 363–365
INDEX694
binary to decimal, 344
broadcasting
definition, 373
distance matrix, 375–377
memory, 374
normalize vectors, 374, 375
rules, 373
complex scalars, 355–357
creation
diagonal matrices, 348
identity matrices, 348
inflating matrices, 353
linspace, 351
logspace, 351
meshgrid, 352
nan() function, 347
np.array() function, 345
ones, 346
random, 350
range() function, 347
stacking, 351, 352
test matrices, 354, 355
triangle matrices, 349
zeros() function, 346
data types, 341, 342
decimal to hexadecimal, 344
definition, 335
dimensions, 371, 373
dtype argument, 68
dtype argument, 68
formatting, 336, 337
import, 66
MATLAB, 66
vs. MATLAB matrices
column major, 339, 340
dimensions, 338, 339
numeric literals, 338
.resize() method, 340, 341
scalars, 339
np.array() function, 67
NumPy module, 335
operations
bitwise, 369
elementwise, 368
Hermitian, 370
multiplication, 370, 371
transpose, 370
submatrices
extraction, 380–383
updation, 380–383
terms of interest
closest term, 386
find() function, 387
longest term, 385, 386
sorted data, 387
values, 383
text files
reading, 359
writing, 360
typecasting, 343
whos command, 67
O
Object-oriented programming (OOP)
class method, 391–394
collisions, 389, 390
computational performance, 388
performance implications, 333
simulation, 388
vectorized solution, 394
open() function, 189
Operating system
computer’s hardware, 322
CPU cycles, 323
NumPy arrays (cont.)
INDEX695
environment variables, 315, 316
independent method, 322
MATLAB, 315, 316
memory, 322, 323
numerical analyses, 322
program iterates, 323
Python, 315, 324
random vector, 325
setenv() function, 316
simulations, 322
TypeError error, 316
P
pack() function, 281
Pandas, 539, 540
Parallel program, 643
pathlib.Path() object, 191
Pickle files, 255–257
pinned, 19
Pivot tables
multilevel aggregation, 558
single-level aggregation, 556, 557
Plotly, 493
Plots
batch mode, 535, 536
editing, 536, 537
line, 493
point, 493
stack, 504, 505
Plotting
areas, 519
Cartopy, 514
exploration, 514
lines, 517, 519
points, 514–516
Point/line plots
date and time, 499, 500
histograms, 502–504
MATLAB, 493, 494
matplotlib commands, 495
metric vs. US customary, 500
Python, 493–495
saving
command, 497
formats, 496
MATLAB, 496
matplotlib’s function, 496
Python, 496
subplots() function, 497, 499
Young’s modulus, 500, 501
PostgreSQL, 286–289, 293–295
prange() function, 598
Prime factors, 618, 620
print() function, 190
purchase_orders, 296
py2mat() function, 299
Python
deployment, 4
distributions, 4
list, 81
.append() method, 90, 91
comprehension, 98
.count() method, 99
creation, 83
empty, 82, 92
.extend () method, 91
extract range, 85–87
find() function, 95, 96
IndexError, 87–89
indexing, 84, 85
.insert() method, 93
length, 83
list comprehension, 97, 98
INDEX696
membership test, 94, 95
merging, 101–103
methods, 81, 82
nested containers, 93, 94
.pop() method, 99, 100
.remove() method, 100, 101
.reverse() method, 106
sorting, 105
subset of items, 98, 99
unmerging, 103, 104
ValueError exception, 96
zipped, 174
modules, importing, 159
multiple computers, 646
substantial cost, 4
sys.path, 174
alias, 175
append(), 176, 177
insert(), 175, 176
tuples
creation, 107
hashable, 107
parentheses, 108
vs. Python lists, 106
wrapping, 108
variables, 81
Python Package Index (PyPI), 62
Python-specific aspects, 576
Pythran, 599
comments, 600
MATLAB, 613
parallel for loop, 601, 603
performance, 603
Q
Qt library, 529
R
range() function, 51
.ravel() method, 640
Read-evaluate-print-loop (REPL), 13
readmatrix() function, 200
read_text() method, 191
Redis, 300, 301, 303–305
Regression
confidence intervals, 453, 454, 456,
457, 460–462
least squares
ordinary, 449, 451, 452
weighted, 452, 453, 457–459
prediction intervals, 453, 454, 456,
457, 460–462
Regular expressions, 195
Remote computer, 638
re.split(), 195
Roots
multivariate, 465, 466
nonlinear equations, 466, 467
univariate, 463, 464
rstrip() method, 190
S
Satellite, 635
save() command, 258
Scalars, 161–163
Scalene, 584, 586
SciPy
coordinate format (COO), 402
definition, 335
eigensolvers, 398, 399
functions, 415
optimization module, 463
scipy.interpolate module, 427
scipy.optimize module, 467
Python (cont.)
INDEX697
sparse module, 401
sparse matrix, 401
scipy.io.loadmat(), 268
Seaborn, 493
Sea surface temperature (SST)
CARTOPY_USER_
BACKGROUNDS, 520
data, 520, 521
earth_veg_index.jpeg, 520
orthographic projection, 522
Second-order differential equation, 635
.send() and .recv() functions, 281
Set operations
MATLAB, unique() function, 109–110
Python, .issuperset() method, 110
iteration/casting, 109, 110
set() function, 109
Simulated annealing
Annealer-derived class, 474
automatic estimator, 475, 477
compute_distances() function, 475
concept, 474
.distance_matrix attribute, 474
energy() method, 474
parameters, 475
requirements, 474
.state attribute, 474
TSP, 440, 446, 477, 478
Singular value decomposition (SVD),
397, 398
sleeper() and add_numbers(), 46
Solver code, 641
Solver function, 638
South American wheat production, 527
add_geometries() function, 524
borders, 524
Cartopy, 523
cases, 525, 526
country names, 523, 524
database, 523
low resolution file, 523
polygonal regions, 524
Sparse matrices
BSR, 411–413
creation
COO, 402, 403
csr_matrix(), 401
dense matrix, 401, 402
dense representation, 403
COO
dimension, 404
index values, 405
indices/values, 405, 406
row/columns, 404
sparse() function, 404
.sum_duplicates() method, 405
CSC, 406, 407
CSR, 406, 407
definition, 401
diagonal (DIA), 411–413
DOK, 410
formats/capabilities, 424, 425
input and output (I/O)
component parts, 416, 417
mat files, 416
pickle files, 418
LIL, 408–410
linear algebra
eigenvalue problem, 422, 423
finite element–based
symmetric, 418
linear equations, 421, 422
matrix summary, 419, 420
matrix-vector product, 420
scipy.sparse.linalg module, 418
Sturm sequence number, 423
INDEX698
MATLAB, 401
SciPy, 401
.split() method, 195
SQLite, 249–252
Stack plots, 504, 505
start_new_order() function, 296
startup directory, 21
startup.m file, 159–161
strftime() function, 141
Strings
byte arrays, 68, 69
character arrays, 68, 69
formatting, 74, 75
operations
append, 71, 72
commands, 70
conversion, 72
length, 71
key, 70
method chaining, 73, 74
repeat, 72
replace characters, 73
regular expressions
purposes, 77
replacing, 79, 80
separation, 78
substrings, 78, 79
splitting, 75, 76
templates, 80, 81
testing
equality, 76
starting characters, 77
substring, 77
trailing characters, 76
struct() function, 299
struct’s pack() function, 225
Structured data
classes, 122, 123
hasattr() function, 122
isfield() function, 122
namedtuple method, 121
SimpleNamespace method, 121
Structured Query Language (SQL),
249, 250
Symbolic mathematics
char attribute, 487
collisions, 486
derivatives, 482
integrals, 483
laplace_transform() function, 487
linear algebra, 484, 486
s.doit(), 487
series, 486
solving equations, 484
SymPy code generators, 481
variables, 482
sys.exc_info() function, 57
T
Tables
columns
adding, 558, 559
deletion, 560, 561
names, 543, 544
types, 543, 544
creation, 550
.describe() method, 545
groupsummary() method, 546
head() function, 546
indexing, 552
.info() method, 545
joins
flavors, 561
food costs, 563
Sparse matrices (cont.)
INDEX699
food type, 561, 562
how= keyword, 562
wxp column, 563, 564
loading
MATLAB’s table, 541
pets.csv file, 540
read_csv(), 543
string/floating-point types, 542
rows
iteration, 555, 556
sorting, 551
size, 543, 544
subsets
all columns, selected rows, 553
all rows, selected columns, 552
filter rows, conditional
operations, 555
selected rows, selected columns,
553, 555
summary() function, 545
tail() function, 546
.value_counts() method, 546
Test matrices
arrow matrix, 415
random sparse
arrays, 413
spy() function, 414
Text files
assignments, 190
binary files, 222–225
CSV
definition, 200
NumPy’s genfromtxt() function, 203
pandas, 201, 202
python module, 200
ini files, 214–216
I/O exceptions, 192
JSON, 213
MATLAB/Python, 189
parsing text
operations, 193
regex’s, 195
re.split(), 195
splitting string, substring, 194
string search, 196–198
stripping whitespace, 193
pathlib, reading corrupted files, 191
reading/writing numeric arrays, 192
read ini file, 218, 219
STDOUT, 189
write ini file, 220, 221
write YAML files, 217
XML
lxml modules, 203
with namespaces, 207–209
pretty-printing, 211
without namespaces, 203, 204, 206
writing XML, 210
YAML, 212
Time
current, 137
definition, 137
module, 137
string formats, 138, 139
tic/toc/%timeit, 140
time.strftime() function, 137
time.time() function, 137
Timezones, 144, 147
.tobytes() method, 281
to_datetime() function, 550
.to_excel() method, 231
.tofile() method, 223
Traveling Salesman Problem (TSP), 474
Traversing generators, 172, 173
Tuples, np.ones() function, 165, 166
Two-dimensional array, 640
INDEX700
U
Unit systems
defining units, 489, 490
MATLAB, 491
metric values, 488
NumPy array, 489
pint/Symbolic Toolbox, 488, 489
Python, 491
Unix epoch seconds, 148
urlread() function, 278
V
Variable assignment
=, 23, 24
in-place update operators, 25
languages, 23
print expressions, 27, 28
walrus operator, 26
Vectorization, 592
Virtual environments
Anaconda, 18, 19
commands, 17
definition, 15
matpy, 16
open source languages, 15
W
which command, 62
While loops, 37
whosmat() function, 259
write() method, 190
X
$max aggregation, 298
Y
yield keyword, 50
Young’s modulus, 500
Z
zoneinfo, 151, 152
zoneinfo module, 144
كلمة سر فك الضغط : books-world.net
The Unzip Password : books-world.net
تعليقات