$ brew install scala
$ brew install sbt

You can use $SBT_OPTS to pass additional JVM options to sbt.
Project specific options should be placed in .sbtopts in the root of your project.
Global settings should be placed in /usr/local/etc/sbtopts

Update the /usr/local/etc/sbtopts by running the command below. This step is optional.

$ echo '-J-XX:+CMSClassUnloadingEnabled' >> /usr/local/etc/sbtopts
$ echo '-J-Xmx2G' >> /usr/local/etc/sbtopts



https://www.youtube.com/watch?v=-xRfJcwhy7A . Learn Scala in one video from a 10+ 

Links

https://nrinaudo.github.io/scala-best-practices/

https://www.youtube.com/watch?v=n-1L0e29F6E . Spark Streaming ПОТОКОВАЯ ОБРАБОТКА ДАННЫХ В РЕАЛЬНОМ ВРЕМЕНИ
https://www.youtube.com/channel/UCR4iuvbk9DCuieR1ADIi0-Q

My Scala code snippets

https://habr.com/ru/post/330816/

https://medium.com/@linas.m/my-favorite-new-features-of-scala-2-13-standard-library-909a89b0da4 .  Latest Scala 2.13

Reactive programming:
https://itnext.io/demystifying-functional-reactive-programming-67767dbe520b?sk=a40cf9bacbc1bafcf505a9f348e844c6

http://fruzenshtein.com/ . Alex Zvolinsky
https://habr.com/ru/company/raiffeisenbank/blog/448128/ Cats
https://habr.com/ru/company/youla/blog/452914/ ML in Scala

https://hub.mybinder.org/user/sbrunk-almond-examples-ysx34sfv/lab .   Jupiter Notebook
https://habr.com/ru/company/mailru/blog/442688/ .  ML and Spark
https://www.youtube.com/watch?v=9AULOshbw9A
http://rcardin.github.io/design/2014/08/28/eat-that-cake.html

https://www.reddit.com/r/scala/comments/b7f1g3/all_videos_of_scalaua_conference_are_now_available/

Scala Fiddle
 
 http://appliedscala.com/ . book
 https://leanpub.com/fpmortals/read .  book Functional Programming for Mortals - ScalaZ
 
Functional Scala
Deep learning in Scala
https://docs.google.com/presentation/d/11SfsSABS9wc6p1hIGyg70BgbALKejY2SRZIieipaTbE/edit#slide=id.g28eabcf0a1_0_564
https://functional.works-hub.com/blog/Functional-Programming-Jargon

https://hackernoon.com/practical-functional-programming-6d7932abc58b

http://www.lihaoyi.com/post/EasyParsingwithParserCombinators.html

Reactive Programming Akka
https://habr.com/company/arcadia/blog/432004/



sealed trait Shape
case class Circle(diameter: Double) extends Shape
case class Rectangle(width: Double, height: Double) extends Shape

An Algebraic Data Type (ADT) is a type that composes sum types and product types to define a data structure. 
In the preceding code, we defined a Shape ADT, which composes a sum type (a Shape can be a Circle or a Rectangle), 
with a product type Rectangle (a Rectangle holds a width and a height).

The sealed keyword indicates that all subclasses of the trait must be declared in the same .scala file. 

Option
https://subscription.packtpub.com/book/application_development/9781788397643/3/ch03lvl1sec25/using-option

The Scala Option type is an algebraic data type (ADT) that represents an optional value. 
It can also be viewed as List that can contain either one or no elements.

sealed trait Option[+A]
case class Some[A](value: A) extends Option[A]
case object None extends Option[Nothing]

The + sign in front of the A type parameter in the Option[+A] declaration means that Option is covariant in A. 
If B is a subtype of A, then Option[B] is a subtype of Option[A]. 
Furthermore, you might notice that None actually extends Option[Nothing] and not Option[A]. 
This is because a case object cannot accept a type parameter. 
In Scala, Nothing is the bottom type, which means that it is a subtype of any other type.  
This implies that None is a subtype of Option[A] for any A.


scala> Vector(1,2).zip(Vector("a", "b", "c"))
res0: scala.collection.immutable.Vector[(Int, String)] = Vector((1,a), (2,b))

Note that the resulting Vector has a size that is the minimum size of the two arguments. 
The last element, "c", is lost, because there is nothing to zip it with!




scala> val it = Vector(1, 2, 3, 4).sliding(2)
it: Iterator[scala.collection.immutable.Vector[Int]] = non-empty iterator

scala> it.toVector
res0: Vector[scala.collection.immutable.Vector[Int]] = 
Vector(Vector(1, 2), Vector(2, 3), Vector(3, 4))

scala> Vector(1).sliding(2).toVector
res12: Vector[scala.collection.immutable.Vector[Int]] = Vector(Vector(1))

sliding(p) creates an Iterator which will produce collections of size p.
Each collection will have a new iterated element plus all the previous p-1 elements. 
Notice that if the collection size n is lower than p, the produced collection will have a size of n.

Next, we iterate through the sliding collections using collect. collect is similar to map: 
it allows you to transform the elements of a collection, but with the added capability of filtering them.
Basically, whenever you want to filter and map a collection, you can use collect instead.
The filtering is performed using pattern matching. Anything that does not match any pattern is filtered out:



scala> val v = Vector(1, 2, 3)
v: scala.collection.immutable.Vector[Int] = Vector(1, 2, 3)

scala> v.filter(i => i != 2).map(_ + 1)
res15: scala.collection.immutable.Vector[Int] = Vector(2, 4)

scala> v.collect { case i if i != 2 => i + 1 }
res16: scala.collection.immutable.Vector[Int] = Vector(2, 4)

Notice that, in the preceding code, we used map(_ + 1) instead of map(i => i + 1).
This is a shorthand notation for an anonymous function. Whenever you use a parameter once in your anonymous function,
you can replace it with _.





Scala: call by value, call by name and lazy calculation
   def f (arg: Int) = println (arg)     // call by value - arg is evaluated at the point of function call
   def f (arg: => Int) = println (arg)  // call by name  - arg is evaluated at the moment of use (lazy, but it is calculated every time)

   Scala keyword lazy: calculated once at the moment of 1st use

High Order Function
// Apply function  n times to a value x
// nTimes (f, n, x)  = f(f(...f(x)) = nTimes(f, n-1, f(x))

def nTimes(f: Int => Int, n: Int, x: Int): Int =
   if (n < 1) x
   else nTimes(f, n-1, f(x))
             
Usage:             
 val plusOne =(x: Int) => x+1
 println(nTimes(plusOne, 10, 1)   

 Let do it differently by returning lambda:
def nTimesBetter(f: Int => Int, n: Int): (Int => Int) =
   if (n < 1) (x:Int) => x
   else (x: Int) => nTimesBetter(f, n-1) (f(x))
             
Usage:             
 val plus10 = nTimesBetter(plusOne, 10)
 println( plus10(1))     

Carried Function
val superAdder: Int => (Int=>Int) = (x: Int) => (y: Int) = x+y
val add3 = superAdder(3) // y => 3+y
println(add3(10))
println(superAdder(3)(10)) 

    Functions with muliple parameter lists:
  def curriedFormatter(c: String) (x: Double): String=c.format(x)
  val standardFormat: (Double=> String) = curriedFormatter("%4.2f")
  val precisionFormat: (Double=> String) = curriedFormatter("%10.8f")
    
  println(standardFormat(Math.PI))
  println(precisionFormat(Math.PI))
    
    
apply()
A simple use of apply is to define it on an Object. This lets you call the Object as if the object itself was a function. 
Here’s an example:

object Greet {
 def apply(name: String): String = {
   "Hello %s".format(name)
 }
}
// I can call apply explicitly if I want:
Greet.apply("bob")
Case classes provide you with an automatically generated apply function on their companion object that you can use like a constructor.
This is very confusing as it looks just like a constructor and quacks like a constructor, but it is not a constructor.


Tuples
up to Tuple22
((0,1,2,3,4),( (5,6,(7,8),9))
Tuple2[Tuple5[Int,Int,Int,Int,Int], Tuple4[Int,Int,Tuple2[Int,Int],Int]]

Reduction operations
foldLeft vs fold

def foldLeft[B] (z: B) (f: (B,A) => B): B
The applies a binary operator to a start value and all elements of this collection or iterator,
going left to right
    
[B] means that the function has a type parameter named B

op: (B, A) => B means that f must be a function that has two parameters of type B and A and returns a value of type B. 
    Since foldLeft is a function that takes another function as an argument, we say that foldLeft is a higher order function.    
    
If we consider a coll collection, foldLeft works as follows:

Step 1: It creates a var acc = z accumulator then calls the op function:
acc = op(acc, coll(0)) 

Step 2: It carries on calling op with each element of the collection:
acc = op(acc, coll(i))

Step 3: It returns acc once it has iterated through all elements of the collection    

The foldLeft is not parallelizable. Explanation is below:

val l=List(1,2,3,4)
val res= l.foldLeft("")(str: String, i: Int) => str+i)
Result: "1234"

val l1=List(1,2)
val l1=List(3,4)
l1.foldLeft(...) // "12"
l2.foldLeft(...) // "34"

Issue: not possible to combine results of 2 foldLeft() outputs above using the same foldLeft()
because the signature of foldLeft() 2nd arg  is a func(String,Int), not (String,String).
The foldLeft is not parallelizable and Spark does not support foldLeft() and foldRight().

def fold(z: A) f: (A,A) => A): A   // this is parallelizable because the input and output types are the same

Aggregate
aggregate[B] (z: => B) (seqop: (B,A) =>B, combop (B,B) => B): B  //parallelizable and can change the return type - supported by Spark!


Reduce and fold
val a = Array(12, 6, 15, 2, 20, 9)
val sum = a.reduceLeft(_ + _)   // same as a.reduceLeft((x,y) => x + y)
a.reduceLeft(_ * _)  // a.reduceLeft(_ min _)   //  a.reduceLeft(_ max _)

The foldLeft method works just like reduceLeft, but it lets you set a seed value to be used for the first element.

map and flatMap
    
val numbers=List(1,2,3,4)
val chars=List('a','b','c','d')
val colors=List("black","white")

val combinations=numbers.flatMap(n => chars.flatMap(c => colors.map(color => "" + c + n + "-" + color)))
println (combinations)
List(a1-black, a1-white, b1-black, b1-white, c1-black, c1-white, d1-black, d1-white, a2-black, a2-white, b2-black, b2-white, c2-black, c2-white, d2-black, d2-white, a3-black, a3-white, b3-black, b3-white, c3-black, c3-white, d3-black, d3-white, a4-black, a4-white, b4-black, b4-white, c4-black, c4-white, d4-black, d4-white)    
The same result is produced by :
val forCombinations = for {
  n <- numbers
  c <- chars
  color <- colors
} yield "" +c + n + "-"+color

println (forCombinations)
   
 
Pattern matching
      
    
scala> Vector(1, 2, 3, 4) match {
  case head +: second +: tail => tail
}
res0: scala.collection.immutable.Vector[Int] = Vector(3, 4)

scala> Vector(1, 2, 3, 4) match {
  case head +: second +: tail => second
}

scala> ("0", 1, (2.0, 3.0)) match {
  case ("0", int, (d0, d1)) => d0 + d1
}
res2: Double = 5.0

scala> "hello" match {
  case "hello" | "world" => 1
  case "hello world" => 2
}
res3: Int = 1

scala> def present(p: Person): String = p match {
  case Person(name, age) if age < 18 => s"$name is a child"
  case p => s"${p.name} is an adult" 
}
present: (p: Person)String
    
Iterator
    
scala> val iterator = (1 to 3).iterator
iterator: Iterator[Int] = non-empty iterator

scala> iterator foreach println
1
2
3

scala> iterator foreach println

scala> 
    
 
Sliding
  
sliding(p) creates an Iterator which will produce collections of size p. 
    Each collection will have a new iterated element plus all the previous p-1 elements. 
    Notice that if the collection size n is lower than p, the produced collection will have a size of n.    
    
 scala> val it = Vector(1, 2, 3, 4).sliding(2)
it: Iterator[scala.collection.immutable.Vector[Int]] = non-empty iterator

scala> it.toVector
res0: Vector[scala.collection.immutable.Vector[Int]] = 
Vector(Vector(1, 2), Vector(2, 3), Vector(3, 4))

scala> Vector(1).sliding(2).toVector
res12: Vector[scala.collection.immutable.Vector[Int]] = Vector(Vector(1))
    
 
Collect
     
 collect is similar to map: it allows you to transform the elements of a collection, but with the added capability of filtering them.
    Basically, whenever you want to filter and map a collection, you can use collect instead. The filtering is performed using pattern matching. 
    Anything that does not match any pattern is filtered out:


scala> val v = Vector(1, 2, 3)
v: scala.collection.immutable.Vector[Int] = Vector(1, 2, 3)

scala> v.filter(i => i != 2).map(_ + 1)
res15: scala.collection.immutable.Vector[Int] = Vector(2, 4)

scala> v.collect { case i if i != 2 => i + 1 }
res16: scala.collection.immutable.Vector[Int] = Vector(2, 4)
Notice that, in the preceding code, we used map(_ + 1) instead of map(i => i + 1). 
This is a shorthand notation for an anonymous function. 
Whenever you use a parameter once in your anonymous function, you can replace it with _.

    
    
    
Reading .tsv file
    
    
getLines() returns Iterator[String]. 
 An iterator is a mutable data structure that allows you to iterate over a sequence of elements.  
Next, we drop the first line , which contains the header, and transforms each line using an anonymous function passed to map.

The anonymous function takes line of type string, transforms it into Array[String] using split, and instantiates a new EquityData object.

Finally, we convert the resulting Iterator[EquityData] into Vector[EquityData] using .toVector. 
This step is very important: we convert the mutable, unsafe, iterator into an immutable, safe Vector    
    
    
import scala.io.Source    
def fromResource(resource: String): Vector[EquityData] =
    Source.fromResource(resource).getLines().drop(1).map { line =>
          val fields = line.split("\t")
          EquityData(
                monthId = fields(0), 
                value = fields(1).toDouble, 
                annualDividend = fields(2).toDouble
          )
    }.toVector