python
Use Snapstream
One of the more challenging aspects I encountered while working with stream processing was the task of performing near-merge or as-of joins on Kafka streams.
Manage Kafka Streams
Processing messages from Kafka is relatively easy, but common errors can easily be avoided by having a better understanding of how Kafka works internally. We’ll process messages from the start of the topic, and see how Kafka handles offsets step by step.
Create Python Executables
When you want to share your software with a non-developer, or you want to run a comprehensive python project without having to bother with the setup, executables are quite useful. And it only takes about 5 minutes to get it done.
Install Python Packages From Azure DevOps
Python Package Index pypi.org is not the only source for downloading python packages. It’s possible to host your own package index.
Install Python Packages on Azure Synapse
This post will be very similar to the last one. But this time, we’ll release our python package to Azure Synapse instead of Databricks. In order to unit test Synapse Notebooks, you’ll have to jump through all sorts of hoops.
Install Python Packages on Databricks
Let’s use the same basic setup as in test python code, then use our knowledge from create python packages to convert our code to a package. And finally we will install the package on our Databricks cluster.
Use Functional Programming In Python
Although the official python documentation has dedicated an in-depth document on Functional Programming, the community does not consider FP techniques best practice at all times Popular FP related functions have been moved to functools. Still, FP concepts can be very helpful, and in this post I will demonstrate some ‘Functional Programming’-related concepts, using python.
Create Python Packages
Why should you care about creating packages? Packages are easy to install (pip install demo). Packages simplify development (pip install -e . installs your package and keeps it up-to-date during development). Packages are easy to run and test …
Enhance Your Python-vscode Workflow
This post covers my personal workflow for python projects, using Visual Studio Code along with some other tools. A good workflow saves time and allows you to focus on the problem at hand, instead of tasks that make you feel like a robot (machines are good for that).
Test Python Code
Writing unit tests should an integral part of delivering software for every developer. Whenever a piece of code is changed, it has the potential to break all other parts. The broken parts can even be discovered in a far later stage, having caused potential damage that is hard to restore.
Manage Virtual Environments
In this post we will take a look at using Pipenv, a dependency manager for python, to boost your python workflow. A virtual environment is an isolated environment in which dependencies for a python project are contained.
Manage Python Versions
This will be a micro-post talking about managing multiple python versions on your machine, using a tool called Pyenv. Depending on your operating system, it may be quite a hassle uninstalling and installing different python versions.
