Rearranged my pinned mastodon hashtags and feeling more focused 😎

#ProgressToday

Learning from #EffectivePandas and #PythonForDataAnalysis.

Recipe for permutating or randomly reordering the rows of a DataFrame or Series:
new_order = np.random.permutation(n)
df.iloc[new_order]
df.take(new_order)
To permutate the cols of a DataFrame, add "axis='columns'" to .take().

Method for selecting a random subset of the rows DataFrame or Series:
df.sample(n=, frac=)
To allow for replacement, add "replace=True" to .sample().

#LearnPython #ProgressToday

Learning from #EffectivePandas and #PythonForDataAnalysis.

Recipe for permutating or randomly reordering the rows of a DataFrame or Series:
new_order = np.random.permutation(n)
df.iloc[new_order]
df.take(new_order)
To permutate the cols of a DataFrame, add "axis='columns'" to .take().

Method for selecting a random subset of the rows DataFrame or Series:
df.sample(n=, frac=)
To allow for replacement, add "replace=True" to .sample().

#LearnPython #ProgressToday

Learning from #EffectivePandas and #PythonForDataAnalysis.

The preferred way to index and filter a Series or a DataFrame is i) with .loc[] indexing on index labels or ii) with .iloc[] indexing on index position integers. Their call signatures are nearly identical:

.loc[rows]
.loc[:, cols]
.loc[rows, cols]

Their strengths come from the increased clarity what we intend to index on and what we intend to select, therefore helping us not be the problem 😂

#LearnPython #ProgressToday

#ProgressToday Continued my way through #EffectivePandas and #PythonForDataAnalysis.

Element-wise transformation of a Series values or an Index labels can be done by feeding a dictionary (for selected elements) or a function (for all elements) into method

.map(dict or func)

Binning can be done with methods

.cut(data, bins or nbins, right=, labels=, precision=)
.qcut(data, quantiles or nquartiles)

.cut() bins the data values, while .qcut() bins the data quantiles.

#LearnPython

Continued my way through #EffectivePandas and #PythonForDataAnalysis.

Element-wise transformation of a Series values or an Index labels can be done by feeding a dictionary (for selected elements) or a function (for all elements) into method

.map(dict or func)

Binning of a Series or column can be done with i) the data values, or ii) the data quantiles:

.cut(data, bins or nbins, right=, labels=, precision=)
.qcut(data, quantiles or nquartiles)

#LearnPython #ProgressToday

#ProgressToday Here are two methods that change the values within a Series or column:

.replace(to_replace=, value=, regex=)
.clip(lower=, upper=)

The former is more general, while the latter is for numerical data types.

Also, when a Series or column has string data, .replace() changes whole strings, whereas .str.replace() sub-strings.

#ProgressToday Finished the sections in #EffectivePandas and #PythonForDataAnalysis on converting the data types of a Series or column. Top methods:

.astype(dtype, copy=, errors=)
.convert_dtypes()
pd.to_datetime()
pd.CategoricalDtype(categories=, ordered=)

The first is a general-purpose one for Python and NumPy types, while the second converts to pandas extension types that support NA.

Before converting data types, be sure to take care of codes for missing data or errors

#ProgressToday Finished going through sections in #EffectivePandas and #PythonForDataAnalysis related to duplicated data and cleaning. It's good that the two important methods apply to all three objects - Series, DataFrame, and Index:
.duplicated(subset=, keep=)
.drop_duplicates(subset=, keep=)
One difference is that the kwarg 'subset=' applies to DataFrame objects only, which can have multiple columns to choose from.

#ProgressToday Finished going over sections in #EffectivePandas and #PythonForDataAnalysis related to handling missing data. Here are useful functions on this topic:
.isna()
.notna()
.dropna(how=, thresh=, axis=)
.fillna(value=, method=, limit=, axis=)
.interpolate(method=, limit=, axis=)

#ProgressToday Spent some time tidying up my recent code. Cleaned up some code snippets from my latest notebooks and put them in separate #Python files so that they can be run in one go. Also added one-liner docstrings to help my future self understand.

These habits should help boost my productivity over time. We'll see!

#ProgressToday: Finished reading chapter 3 "Pythonic Syntax and Common Pitfalls" in #MasteringPython. The common pitfalls didn't surprise me as much now. Learned of the "walrus operator" and "switch statement". Installed #pycodestyle, tried it on a couple of my python files, and fixed the issues to get a clean pass.

Going forward, I'll do these:

I. Use pycodestyle to scan my python files as I work on them;

II. Look for opportunities to use "walrus operator" and "switch statement".

#Python

#ProgressToday: Finished reading chapter 10 "Testing and Logging" in #MasteringPython. Used #Python doctest and logging modules a little as in the examples. Decided to take the minimalist approach here:

I. Start using logging module with basicConfig in my scripting;

II. Look for opportunities to practice doctest also in my scripting.

#ProgressToday: Finished reading chapter 9 "Documentation" in #MasteringPython. Used Sphinx with a simplest project and gained a little experience of current, real-world documentation. Decided I'll go a minimalist approach:

I. For my external-facing functions or classes , I'll write at least a one-liner docstring to start with.

II. For my Jupyter notebooks, in addition, I'll use markdowns to comment on requirements, input data, data cleaning, plots, etc.

#Python

#ProgressToday: Installed Python 3.11, created a venv, and installed Jupyter in it. Keep going!

#ProgressToday: Finished working through the 25 recipes of chapter 8 "Classes and Objects" in #PythonCookbook. While some concepts were hard to appreciate at first, e.g. descriptors, revisiting them for a second time made the difference. Also, chapter 7 "Classes and OOP" in #PythonDistilled helped me grasp the concepts from a higher level. It's good to learn different ways of defining classes and advanced ways of customizing and optimizing them. I look forward to using these recipes! #Python

I did a quick #introduction on 11/6, and feel the need to do a slow one.

My wife and I are raising a 4YO and I toot #parenting, #ParentingJoy, #Preschooler to share with other parents.

I'm a practicing pythonista (#Python) and working my way through #PythonCookbook and #PythonDistilled. I toot #ProgressToday to record my progress. Right, maybe you didn't know - You are my accountability partners. Thanks!

I enjoy good humor. Sometimes my reply doesn't make sense, hopefully it makes you smile.

#ProgressToday: Finished reading chapter 7 "Classes and OOP" in #PythonDistilled and worked through ~1/3 of the recipes of chapter 8 "Classes and Objects" in #PythonCookbook. Feeling updated in the relevant concepts and patterns in #Python. While more advanced ideas are still brain bending, simpler ones are starting to make sense.

BTW, the above was yesterday's summary, which seems to have been lost during the @sfba.social server team's attempted upgrade to Mastodon v4 last night.

#ProgressToday: Finished reading chapter 7 "Classes and OOP" in #PythonDistilled and worked through ~1/3 of the recipes in chapter 8 "Classes and Objects" in #PythonCookbook. Feeling updated in a number of #Python concepts and patterns with respect to classes and objects. While some more advanced ones are still brain bending, some simpler ones are already making sense. Good to make progress! 😎