Spark 1.6.2, Java code (sorry), this will change every instance of Tesla to S for the entire dataframe without passing through an RDD: dataframe.withColumn(“make”, when(col(“make”).equalTo(“Tesla”), “S”) .otherwise(col(“make”) ); Edited to add @marshall245 “otherwise” to ensure non-Tesla columns aren’t converted to NULL.
Instead of playing whac-a-mole by listing specific tasks to disable (publish, publish-local, publish-signed, etc), another option is to turn off artifact publishing at the source. publishArtifact := false Even though there’s no publishing happening, I also found I needed to supply a publishTo value to make sbt-pgp’s publish-signed task happy. It needs this value, even … Read more
If you have x as Option[Long], x.get will give you Long.
What you observe is expected behaviour – yet of course it’s very bad. Good that known solutions and best practices exist to guard against it. In this answer I’d like to spend some time to explain the issue short, long, and then in depth – enjoy the read! Short answer: “don’t block the routing infrastructure!“, … Read more
First there is a plethora of HTML scraping libs in JVM all you need to do is pimp one of them (pimp my library pattern). The four I have used are: HtmlUnit – Will emulate the browser and even run Javascript Jericho – Preserves formatting and ideal if you want to edit the scraped HTML … Read more
The problem with the Application trait is actually described in its documentation: (1) Threaded code that references the object will block until static initialization is complete. However, because the entire execution of an object extending Application takes place during static initialization, concurrent code will always deadlock if it must synchronize with the enclosing object. This … Read more
+: works with any kind of collection, while :: is specific implementation for List. If you look at the source for +: closely, you will notice that it actually calls :: when the expected return type is List. That is because :: is implemented more efficiently for the List case: it simply connects the new … Read more
But it seems Scala Arrays & Lists doesn’t provide any methods for adding items dynamically due to the immutable nature. Well, no. Scala Arrays are just Java arrays, so they are mutable: val arr = Array(1,2) arr(0) = 3 // arr == Array(3, 2) But just as Java (and C/C++/C#/etc.) arrays, you can’t change the … Read more
Then you’d have to encode the message type into the Actor ref, which would drastically decrease the value of something like the ActorRegistry. Also, with powerful mechanics like “become” (which is fundamental to the actor model) typing the messages is less valuable. Since Akka doesn’t leak memory when a message is not matched to the … Read more
Spray is production ready, but the development team (Mathias Doenitz) works for Typesafe on Akka-http now. The status of Akka-http is “development preview”. There are vague promises of a full release “within a few months”, but nothing you can take to the bank. Edited 29-July-2015: The status of Akka-HTTP is now “release candidate” with version … Read more