Python

python -m SimpleHTTPServer <8000> (python2)

python3 -m http.server <8000> (python3)

http://192.168.1.2:8000

http://127.0.0.1:8000

https://github.com/gto76/python-cheatsheet http://grishaev.me/interview/ https://julien.danjou.info/

CPython compiles the python code into the byte code (myfile.py → mifile.pyc). Then the python interpreter consumes the byte code and act on it.
For performance reasons Python doesn't recompile every time, so it caches the content of the compiled code.

https://indianpythonista.wordpress.com/2018/01/05/demystifying-pyc-files/

Python can be invoked from the command line with several flags; you can find the flags via python --help

https://dbader.org/blog/records-structs-and-data-transfer-objects-in-python

https://github.com/vinta/awesome-python

All pairs similarity search

https://github.com/ekzhu/SetSimilaritySearch/blob/master/scripts/all_pairs.py https://news.ycombinator.com/item?id=18483859

slots

https://habr.com/post/427909/ . slots https://blog.usejournal.com/a-quick-dive-into-pythons-slots-72cdc2d334e http://book.pythontips.com/en/latest/__slots__magic.html

Modules and Packages

Big application usually contains many python files, the import statement allows to glue the code. import mod When the interpreter executes the above import statement, it searches for mod.py in a list of directories assembled from the following sources: The directory from which the input script was run or the current directory if the interpreter is being run interactively The list of directories contained in the PYTHONPATH environment variable, if it is set (not recommended) An installation-dependent list of directories configured at the time Python is installed If a file named init.py is present in a package directory, it is invoked when the package or a module in the package is imported. The resulting search path is accessible in the Python variable sys.path which you can print: python -c "import os, sys; print(os.linesep.join(sys.path))" https://realpython.com/python-modules-packages/

Package managers: pip and conda

pip check . # test consistency of current setup https://snarky.ca/why-you-should-use-python-m-pip/ https://news.ycombinator.com/item?id=21429250 pipx

https://jacobian.org/2019/nov/11/python-environment-2020

https://towardsdatascience.com/how-to-setup-an-awesome-python-environment-for-data-science-or-anything-else-35d358cc95d5

PIP package manager: https://pypi.org/project/pip/ https://dev.to/elabftw/stop-using-sudo-pip-install-52mn The 3rd-party libraries installed by pip are usually here:

python -c "import os, site; print(os.linesep.join(site.getsitepackages()))"

Why you should use python -m pip , not just pip https://habr.com/ru/company/otus/blog/475392/ python3.8 -m pip

pip commands:

pip -V # shows pip version and path (make sure it in sync with output of site.getsitepackages()) above)
pip list                                    # all installed packages
pip show <packagename>     # the package details
pip install <packagename>
conda is similar to pip - it goes with Anaconda Python distribution https://conda.io/docs/user-guide/overview.html

sys.version_info

Enforcing named parameters

def fn(*, a = 100,b = 200):
    return a+b*10

Variadic Functions

def fn(a ,*all ):
    sum=0;
    for item in all:
        sum+=item
    return a+sum
    
print(fn(10,20)) # 30

Keyword Parameters

def fn(**kwargs ):
    return kwargs['a'] + kwargs['b']

print(fn(a=200,b=500))
d1 = {'a':100 , 'b':200}
print(fn(**d1)) # 300

To change global variables

num = 9
 
def f1():
    global num
    num = 20

Closure

one can pass a function as a parameter to another function or return a function from another function.

def getmulby(m):
    def op(n):
        return m*n
    return op
 
f1=getmulby(10)
f2=getmulby(5)
 
print( f1(2) ) # 20
print( f2(2) ) # 10   

Decorator

Decorator is a good example of closure. Decorator is a function that takes another function and extends the behavior of the latter function without explicitly modifying it

   def add_stars(some_function):
    def wrapper():
        print("********************")
        some_function()
        print("********************")
    return wrapper
 
@add_stars
def my_function():
    print("Hello!!!")
 
my_function()

Output: Hello!!!

list.sort() vs sorted()

nums = [1, -3, 2, 5]
new_nums = sorted(nums)
print(new_nums)  # -3, 1, 2, 5
print(sorted(nums, reverse=True))  # 5, 2, 1, -3
# abs is the built in absolute value function
print(sorted(nums, key=abs))  # 1, 2, -3, 5


nums = [1, 2, 3]
for i in reversed(nums):    # <- reversed
  print(i)  # 3, 2, 1  
 
# sum() min() max()
def average(nums):
  return sum(nums)/len(nums)

def range(nums):
  return max(nums)-min(nums)

# Enumerate()
for index, num in enumerate(nums):
 
# Sort the words in the string based on length:
 sorted(s.split(), key=len)

# Join list elements into string:
words = ["Hello", "World"]
combined = " ".join(words)

Lambda Expressions

ls = [2,4,6]
 
newlist = map(lambda item:item * 2, ls)
 
for n in newlist:
    print(n)  

Virtual Environments

https://github.com/pipxproject/pipx
mksir ~/.venvs
To create:
python3 -m venv ~/.venvs/myproject
To activate:
. ~/.venvs/myproject/bin/activate

Python virtual env Python virtual environment

When you create a virtual environment, all you are doing is creating a copy of the Python executable, some core modules in the dist-packages folder, and (usually) an empty site-packages folder. That it.

python -c "import sys; print(sys.executable)"

Using virtual environments allows you to avoid installing Python packages globally which could break system tools or other projects. There are several tools to manage the virtual environments: https://stackoverflow.com/questions/41573587/what-is-the-difference-between-venv-pyvenv-pyenv-virtualenv-virtualenvwrappe/41573588#41573588

https://virtualenv.pypa.io/en/stable/ By default, Virtualenv creates virtual environment using the version of Python under which it is installed. To specify another version of the Python use the -p option. On some occasions, you might want to create a virtual environment with the packages from the global Python installation. This can be accomplished using --system-site-packages option. virtualenv v1 # create new virtual environment named v1; it will have the local folders: bin, include and lib

source v1/bin/activate      # activate it; it changes the $PATH so that the bin/directory of the virtual environment will become first in the list
echo $PATH                         # it is different now
which python
pip install ...                        # packages will be installed into v1/lib/pythonX.Y/site-packages 
deactivate                           # deactivate 

pyenv

https://github.com/pyenv/pyenv https://axcoto.com/notes/manage-python-versions-on-macosx/ https://www.marc-richter.info/using-pyenv-to-manage-your-python-interpreters/

https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html conda-env list

conda environments:

base * /Users/miclub01/anaconda3 tensorflow /Users/miclub01/anaconda3/envs/tensorflow tensorflow_env /Users/miclub01/anaconda3/envs/tensorflow_env

conda activate tensorflow

http://pycon.ru/ My Python code snippets

https://python-patterns.guide/ https://habr.com/company/it_people/blog/422363/ Videos from Russian Python Conf https://github.com/faif/python-patterns

Memory management

https://habr.com/ru/company/otus/blog/443312/ .
https://habr.com/ru/post/455722/

https://pythonprogramming.net https://pythonawesome.com/ https://pythondigest.ru/ https://pynative.com/python-postgresql-tutorial/ Postgres https://rushter.com/blog/numba-cython-python-optimization/ https://hackernoon.com/python-tricks-101-2836251922e0 https://habr.com/post/421993/

Asyncio

https://realpython.com/async-io-python/ https://habr.com/ru/post/453348/ https://youtu.be/pIXiChn5j4E https://www.techiegeek.eu/?p=25

http://theautomatic.net/2017/09/29/downloading-every-file-ftp-server/

https://www.youtube.com/watch?time_continue=187&v=WiQqqB9MlkA . Nina Zakharenko - Elegant Solutions For Everyday Python Problems - PyCon 2018

alex.dzyoba.com/blog/python-import/ import

https://dwhsys.com/2017/01/23/working-with-multiple-python-versions/

    pip check
    python -c "import os, sys; print(os.linesep.join(sys.path))"  

How to see where module is located in file system:

    import six
    print six.__file__

PYTHONPATH

In order to let python search first the most updated version of certain package, instead of removing the system version, what can be done is to set the system variable PYTHONPATH in the ~/.bash_profile (or ~/.bashrc if linux) config file to the path where the new packages are installed:

export PYTHONPATH=/Library/Python/2.7/site-packages

An alternative is to modify the python path inside your python script by adding the path at the beginning of the path list:

import sys
sys.path.insert(1,'/Library/Python/2.7/site-packages')

This needs to be done for every script you need a certain package version. You might want for some reason use an older version that you have installed.

Random notes

assert hasattr(Base,'foo'), "no method foo in Base class"

Можете запустить ее предварительно — strace python script.py — но обычно удобнее подключаться к уже работающему приложению: strace -p PID.

Algorithms in Python Python cheatsheet https://habrahabr.ru/post/349860/ regexp

http://overapi.com/python https://opensource.com/article/17/4/grok-gil https://www.youtube.com/watch?v=7lmCu8wz8ro https://www.youtube.com/watch?v=MCs5OvhV9S4 Python concurrency https://www.amazon.com/Python-Data-Analysis-Wrangling-IPython/dp/1491957662 2nd edition https://www.amazon.com/Python-Tricks-Buffet-Awesome-Features/dp/1775093301 https://www.amazon.com/Dynamic-Programming-Coding-Interviews-Bottom-Up/dp/1946556696 https://www.hardikp.com/2017/12/30/python-cpp/

HTTP Server

How to start http server:

python2 -m SimpleHTTPServer python3 -m http.server 8000

http://192.168.1.2:8000 http://127.0.0.1:8000

If the directory has a file named index.html, that file will be displayed. If there is no index.html, then the files in the directory will be listed. If you wish to change the port from default (8000) then specify it explicitly:

python -m SimpleHTTPServer 8080

http://arseny.info/2017/parallel-computation-with-two-lines-of-code.html Joblib https://www.dataquest.io/blog/python-counter-class/ https://github.com/ChrisKnott/Eel offline web based UI for python

a='12345' a[::-1] # reverse string a[-1] # 5 - last item in the array equivalent to slice(-1) a[-2:] # 45 - last two items in the array equivalent to slice(-2) a[:-2] # 123 - everything except the last two items equivalent to slice(None, -2)

List traversal

range(start, stop, hop)
range(n) # [0,1,...,n-1]
range(1,n) # [1,...,n-1]
range(1,n,2) # [1,3,5,...,n-1] if n is even, or [1,3,5,...,n-2] if n is odd
range(n,-1,-1) # [n,n-1,n-2,...,0]
range(len(arr)) # Provides indices of an array arr
range(len(arr)-1,-1,-1) # Provides indices of arr backwards

List slicing

arr[w:s] # Wait w elements, start copy (:), stop before reaching index s
arr = [1,2,3,4]
arr[1:] = [2,3,4]
arr[:2] = [1,2]

List manipulation

arr = [1,2,3]
[str(x) for x in arr] # Output: ['1','2','3']
map(lambda x: str(x), arr) # Output: ['1','2','3']
[str(x) for x in arr if x%2] # Output: ['1','3']

List as queue

arr = [1,2,3]
arr.append(x) # queue.push(x)
arr.pop(0) #queue.pop() .  removes the 1st element from array
arr[0] #queue.peek()

List as stack

arr = [1,2,3]
arr.append(x) #stack.push(x)
y = arr.pop() # stack.pop() . removes the last element from list
arr[-1] # stack.peek()

Dictionary

    for k, v in dict.items():
	    print(k,v)

d.keys()   d.values() . d.items() .  this is Dictionary Views in Python 3
d.iterkeys() . d.itervalues() d.iteritems()

Default dictionary

from collections import defaultdict
d = defaultdict(list)
d['python'].append("awesome")
d['something-else'].append("not relevant")
d['python'].append("language")
for i in d.items():
    print i

Match brackets

expect = {"(":")","[":"]","{":"}"}

def wellformed(string):
    stack = []
    for char in string:
        if char in expect.keys():
            stack.append(char)
        elif char in expect.values():
            if not stack or expect[stack.pop()] != char:
                return False
    return not stack

print(wellformed("([])[]({})")) # True
print(wellformed("([)]")) # False
print(wellformed("((()")) # False

multiply a vector by a scalar:

def scale(A, x): return [ai*x for ai in A]

add 2 vectors:

def add(A, B): return [ai+bi for (ai, bi) in zip(A, B)]

dot product: def dot(A, B): return sum([ai*bi for (ai, bi) in zip(A, B)])

matrices by vector multiplication:

def mul(A, X): return [dot(ai, X) for ai in A]

Counter usage

from collections import Counter

def is_anagram(word1, word2):
    return Counter(word1) == Counter(word2)

print(is_anagram('tachymetric', 'mccarthyite'))
print(is_anagram('banana', 'peach'))

print most frequent words in file

import re
from collections import Counter

def extract_words(filename):
    text = open(filename).read().lower()
    return re.findall('\w+', text)

def filter_words(words, ignore=None):
    if not ignore:
        ignore = set(['is','if','a','it','the','an','in','of','to','and','that','be',
              'his','he','her','on','not','by','s','ch','are','this','as','for',
              'was','with','which','or','for','from','i','you','at','when','have',
              'but','may','they','their','be','who','your','says','said','all',
              'him','1','2','3','4','5','6','7','8','9','t','o','-','_'])

    return [word for word in words if word not in ignore]

def count_words(words, limit=10, ignore=None):
    filtered = filter_words(words, ignore)

    top_words = Counter(filtered).most_common(limit)
    return top_words

def show_top_words(filename, limit=10):
    words = extract_words(filename)
    for word, frequency in count_words(words, limit):
        print("%15s : %s" % (word,frequency))

if __name__ == "__main__":
    filename = sys.argv[1]
    limit = int(sys.argv[2])
    show_top_words(filename, limit)

Polinom

 class Polynom:
     def __init__(self, *coefs):
         self.coefs=coefs

     def __call__(self):
        ....

     def __repr__(self):
         return "Polynom (*{!r})".format(self.coefs)
     def __add__(self, other):
         return Polynom(* (x+y) for x,y in zip(self.coefs, other.coefs))

--  dictionary -----

 a={'k1':'v1', 'k2':'v2'}
 for key, val in a.iteritems():
    print key, val


 fruits = { 'apple':1,  'orange':2, 'banana':3   }
  if 'apple' in fruits:
        print(fruits['apple'])


--- Exceptions ---
def linux_interaction():
    assert ('linux' in sys.platform), "Function can only run on Linux systems."
    print('Doing something.')

try:
    linux_interaction()
except AssertionError as error:
    print(error)
else:
    try:
        with open('file.log') as file:
            read_data = file.read()
    except FileNotFoundError as fnf_error:
        print(fnf_error)
finally:
    print('Cleaning up, irrespective of any exceptions.')


--  Fibonacci ---------

def fib(num):
  a,b=0,1
  for i in xrange(0,num):
      yield "{}: {}".format(i+1,a)
      a,b = b, a+b

for item in fib(10):
   print item