Where are logs in Spark on YARN?

You can access logs through the command yarn logs -applicationId <application ID> [OPTIONS] general options are: appOwner <Application Owner> – AppOwner (assumed to be current user if not specified) containerId <Container ID> – ContainerId (must be specified if node address is specified) nodeAddress <Node Address> – NodeAddress in the format nodename:port (must be specified if … Read more

Buiding Hadoop with Eclipse / Maven – Missing artifact jdk.tools:jdk.tools:jar:1.6

The problem is in the Eclipse Maven support, the related question is here. Under Eclipse, the java.home variable is set to the JRE that was used to start Eclipse, not the build JRE. The default system JRE from C:\Program Files doesn’t include the JDK so tools.jar is not being found. To fix the issue you … Read more