Tuesday, September 8, 2020

How do implicit work in Spark/Scala ?

Q: 

Here is a sample Spark code, which converts the Seq to the Dataset:

import spark.implicits._
val s = Seq(1, 2, 3, 4)
val ds = s.toDS()

The Scala Seq does not have the toDS method, it comes from Spark implicits. How is the Dataset created here?

A:

Scala has a way to add methods to existing classes, like extension method in Kotlin (and C# as I remember), but does it in a different way, through implicits.

To add the method to existing class, you first create implicit class:

object StringImplicits {
  implicit class StringUtils(s: String) {
    def someCoolMethod = println("Yooo")
  }
}

object Application extends App {
    import StringImplicits._
    val s = "Hello"
    s.someCoolMethod
}

You import this StringUtils and can call someCoolMethod on instance of String

Notice that StringUtils class takes String as a constuctor param.

When calling some method on String, scala compiler first looks this method in String class.

If it does not find it, it will look imported implicit classes which take String param.

If found, it calls the method from that class.

If no such class found, it will raise the error.


Monday, September 7, 2020

PDF to word conversion software?

 Download Abiword from Ubuntu Software Center or you can install it by typing following command in terminal:

sudo apt-get install abiword

Then perform the conversion:

abiword --to=doc example.pdf
OR

Install AbiWord from Ubuntu Software Center

Open Pdf Files with it.

Use Save As.. to save pdf in Word Doc format.

Its this easy :)

Friday, September 4, 2020

Formula to Read and understand properly and remembers more time, Revision at some intervals , Analytical Skills , Problem Solving Skills

Formula to Read  and understand properly and  remembers more time, Revision at some intervals , Analytical Skills , Problem Solving Skills 


I => Imagination - Imagine everything.(if image and source are mostly same then we can consider source as image, Image may be clear image or if clear image is not there then we will create some image for as of now. Letters are also some short of images. Imagination will be Clear Image and  Letter Images combination Or Only Clear Images Or Only  Letter Images )

L=> Linking the images (If we miss the linking between the images , then we we can not explain very well.)

P=> Identify the positives

N=>Identify the negatives

R=> Real Life Usage - Need to get at least one real life usage . 

N=> Notes

  • This is helpful to revision and to explain to others(if possible we must draw our image and show them to understand better.
  • If most((90%-mind will determine this) of the point of a topic are unknown or Image linking is missed then we can draw the images(if we have more time or possible to draw )
  • if Our Image and the Real image is same then need not to re draw the same again(wish).

E => Excellent but Not Enough => Try to improve the existing .

T=> Teaching to others with the Drawn Imaginative Linked Images.

Revision at some intervals (Hour wise , day wise  and month wise)


Practice Practice Practice........


Reading Skills : Imagination, Linking images, Positives, Negatives , Real time usage,Notes,Teaching

RevisionSkimming , Scanning ,Detailed Revision on any topic (mind decide)

Analytical skill is the ability to deconstruct information into smaller categories in order to draw conclusions (with the help of problem solving skill).

Problem solving skill is to identify all the participants and understand each participants positives and negatives and apply design thinking (create more ideas and pick the best one , create prototype and test it)








Recent Post

Databricks Delta table merge Example

here's some sample code that demonstrates a merge operation on a Delta table using PySpark:   from pyspark.sql import SparkSession # cre...