Troubled by confusing concepts such as Executors, Node, RDD, Task in spark? Invest just 2 minutes of your time to make some order in this mess!

I'll clean up these apache spark concepts for you!

Spark building blocks: executor,tasks,cache,sparkcontext,cluster manager

Executor => Multiple Tasks: is a JVM process sitting on all nodes.  Executors receive tasks (jars with your code) deserialize it, and run it as a task.

Executors utilize cache so that the tasks can run faster.

Node => Multiple Executors: Each node has multiple executors.

RDD => Big DataStructure: Its main strength is that it represents data which cannot be stored on a single machine, so its data is distributed, partitioned, split across computers.
12

You see it's much easier than you think there exists a limit set of rules you should apply to most of the programming interview questions which involves algorithms and data structures. I have prepared a summary of them for you, just read below and get your tips for today.

When you have no clue / Under panic attack => Brute Force!

If you don't have a clue, brute force the fu**** question! In most cases the question you are presented with has a brute force solution.

Below is a list of topics to study for cs interview.  If you have any comments please let us know.

The topics include data structures, sorting, search, graph search, math, compression, security, web, recursion, general programming, data science: kafka, hadoop, storm, UML, java, scalability, multithreading.

For each topic we have a status column, use it for our own to track the status of your progress in the study this topic.

Scalability CheatSheet— Part 3 — PAXOS

We like journaling, seriously, it helps us avoid data corruption you could update a data and fail really — i mean there could be an electricity shutdown whatever, this is why we like journaling it’s append only, so nothing can really be corrupted except for what you append, but if it’s corrupted you don’t consider it as appended.

Introduction

Here are a few guidelines for supporting scalability and performance in your systems.

Simplify — Simplify your code and design, you will gain from it an easier to understand and a scalable system, your life will be scalable, the more complex it is the less it’s possible to scale it out and the more complex your life is.
Popular Posts
Popular Posts
Contributors
Contributors
Archive
Archive
Labels
Labels
Loading
Dynamic Views theme. Powered by Blogger. Report Abuse.