cluster-computing – Row Coding

What’s the meaning of “Locality Level”on Spark cluster

July 25, 2023 by Tarik

The locality level as far as I know indicates which type of access to data has been performed. When a node finishes all its work and its CPU become idle, Spark may decide to start other pending task that require obtaining data from other places. So ideally, all your tasks should be process local as … Read more

Docker-Swarm, Kubernetes, Mesos & Core-OS Fleet

October 18, 2022 by Tarik

Disclosure: I’m a lead engineer on Kubernetes I think that Mesos and Kubernetes are largely aimed at solving similar problems of running clustered applications, they have different histories and different approaches to solving the problem. Mesos focuses its energy on very generic scheduling, and plugging in multiple different schedulers. This means that it enables systems … Read more