The JVM Fanboy Blog 

Entries tagged [building]

Host Maven repositories on a Raspberry Pi

by vincent


Posted on Sunday May 08, 2016 at 11:42PM in General


For a particular project I'm doing, it would be handy to re-use a JAR file in multiple other future projects. Although there are far more simple solutions to this problem, when I read about repository managers in the official Maven documentation, I gave it some thought and decided to try to run one on one of my Raspberry pi mini-computers that was waiting for a problem to be solved :-)

When hosting a Maven repository on the Raspberry pi, next to hosting private repositories that can be accessed by other computers in my network, it can act as a proxy for Maven Central as well. This means that when a computer in my network needs to download a dependency from Maven Central, it will get a cached copy from the Raspberry pi instead. If the dependency is not already in its cache, it will download the dependency from Maven Central (if it is configured to allow this) first.

Apache Archiva

Looking at the list of available Maven Repository Managers, Apache Archiva caught my eye and decided to give it a try.

Like many other Apache Software Foundation projects that I follow, Archiva is a project with light traffic and not a lot of new releases lately, this is not necessarily a bad thing though, hopefully this is a sign that the current version is generally considered stable.<.p>

What I saw when looking up for more information, impressed me. The web user-interface looks really user-friendly and nice.

Browsing repository in Archiva

Running Archiva in Standalone mode or Servlet container?

Archiva can run in two modes:

  • Stand-alone (a script can be started on the console, that will start an embedded web-server)
  • Run inside a Servlet container (Apache Tomcat) or dedicated Java EE application server (Glassfish, JBoss/Wildfly, Oracle WebLogic...)

Let's take a look at both options

Standalone mode on the Raspberry pi

At first I tried running Archiva in stand-alone mode, assuming this would be ideal for the Raspberry pi.

I had many issues along the way, which all had to do with Archiva's dependency of an out-dated version of Java Service Wrapper by Tanuki Software. As I understand Service Wrapper is a component that lets Java applciations run as Linux daemons and Windows services. Due to licensing issues, the Archiva team can not upgrade this component to a more up to date version.

Blogger Ti-80 - almost exactly three years ago - made a blog post about running Archiva on the Raspberry pi and it seems all problems can be solved by manually building the project and replacing binaries.

I decided to not follow this route and run it in a servlet container. I reasoned I'll probably want to run additional servlets on this Raspberry pi in the future anyway. And Adam Bien convinced me that Tomcat's memoy consumption is not as excessive as some people seem to think.

Running inside Servlet Container

Here are the steps that I took to get this up and running. I did not follow the documentation exactly, as I have some other conventions. Feel free to disagree with me and use their instructions! Note that I use a Raspberry pi 2 (4 cores and 1GB of internal memory), I have not tried this on other models.

The official installation guide is here.

  • Login with SSH to your Raspberry pi, I used the built-in pi user (you did change its default password, didn't you?! :) )
  • Create a dedicated Linux user for running Tomcat:

    sudo -s
    adduser tomcat

    Follow the prompts, then change active user to "tomcat"

    su tomcat

  • Let's create directories and download Tomcat and Archiva

    cd ~
    mkdir downloads
    cd downloads

    Visit the TomCat homepage and look for the download link of the .tar.gz release of the currently stable version. At the time of this writing this was 8.0.33

    wget URL-TO-TOMCAT.tar.gz
    (replace "URL-TO-TOMCAT.tar.gz" with the download URL)
    tar xvfz ./apache-tomcat-X.Y.Z.tar.gz
    (replace "apache-tomcat-X.Y.Z.tar.gz" with downloaded file)
    mv ./apache-tomcat-X.Y.Z ~/tomcat
    (move the created directory, not the downloaded tar.gz file!, to the home directory)

  • Visit the Archiva homepage and look for the download link of the WAR release of the currently stable version. At the time of this writing this was 2.2.0

    wget URL-TO-ARCHIVA.war (replace "URL-TO-ARCHIVA.war" with the download URL)

  • I create a dedicated directory for Archiva and do not place it in the Tomcat home directory (which the Installation Guide suggests). I don't want to pollute Tomcat home directory with Archiva related files. This is a debatable decision, as some dedicated TomCat configuration files will be required to run Archiva anyway.

    cd ~
    mkdir -p webapps/archiva
    cd webapps/archiva
    mkdir conf
    mkdir db
    mkdir logs
    mkdir war

    cd war
    cp ~/downloads/*archiva* ./

  • Let's create Tomcat context configuration XML file.

    cd ~/tomcat/conf
    mkdir -p Catalina/localhost
    cd Catalina/localhost
    nano archiva.xml
    (Install nano if it could not be found: apt-get install nano)

  • Enter following code in Nano
    (Substitute the docBase path with the full path to your downloaded war file)

    <?xml version="1.0" encoding="UTF-8"?>
     <Context path="/archiva"
              docBase="/home/tomcat/webapps/archiva/war/apache-archiva-2.2.0.war">
    
     <Resource name="jdbc/users" auth="Container" type="javax.sql.DataSource"
               username="sa"
               password=""
               driverClassName="org.apache.derby.jdbc.EmbeddedDriver"
               url="jdbc:derby:/home/tomcat/webapps/archiva/db/users;create=true" />
    
     <Resource name="mail/Session" auth="Container"
                type="javax.mail.Session"
                mail.smtp.host="localhost"/>
     </Context>
                

    Press CTRL+X , then Y to exit.

  • A bit annoyingly, you'll need to install some dependencies in the "lib" directory of Tomcat

    You could download the Archiva .zip release and get it from them, but you could also download the files from mvnrepository.com, what we'll do here. I've chosen the exact versions used by the current Archiva stable version to prevent conflicts. We could regret this later, when installing applications that require newer versions... :( .

    Check the installation guide to see if the version numbers mentioned here still match with the latest version!

    cd ~/tomcat/lib

    Vist http://central.maven.org/maven2/javax/activation/activation/1.1/activation-1.1.jar and find the "Download JAR" button. Copy the link URL.

    wget LINK-TO-ACTIVATION-1-1.JAR (replace with full link to activation-1-1.jar)

    Vist http://mvnrepository.com/artifact/javax.mail/mail/1.4 and find the "Download JAR" button. Copy the link URL.

    wget LINK-TO-MAIL-1-4.JAR (replace with full link to mail-1.4.jar)

    Vist http://mvnrepository.com/artifact/org.apache.derby/derby/10.12.1.1 and find the "Download JAR" button. Copy the link URL (manual states newer versions of Derby than mentioned in the documentation should work fine).

    wget LINK-TO-DERBY.JAR (replace with full link to Derby jar file)

  • Finally create a start script that will boot Tomcat and additionally sets some required environmental variables.

    cd ~
    nano startup.sh

    Add the following content:

    export CATALINA_OPTS="-Dappserver.home=/home/tomcat/webapps/archiva -Dappserver.base=/home/tomcat/webapps/archiva"
    /home/tomcat/tomcat/bin/startup.sh
             

    Press CTRL+X , then Y to exit.

    chmod +x ./startup.sh

  • Run the script:
    ./startup.sh
  • On the computer you used to log in to Raspberry pi, start your browser (so do not run the browser on your Raspberry pi), go to:
    http://IP_ADDRESS_RASPBERRY_PI:8080/archiva
    (replace IP_ADDRESS_RASPBERRY_PI with the correct ip address or host name)

    If everything goes well, after a few seconds you should see a welcome screen, with a button at the top right side to create an admin user. Note that booting can take some time.

    On problems, you'd get a simple 404 screen. In that case you'll have to look at the log files of Tomcat and try to determine what is wrong, usually something's wrong with one of the paths or dependencies:
    nano /home/tomcat/tomcat/logs/localhost.YYYY-MM-DD.log (replace with your current date)

  • Create the admin user and follow the prompts.
  • I should do a future blog post about configuring Archiva, I feel the default settings are good enough to get started.

Configure Maven clients to use Archiva

To get started, let's configure your desktop machine clients to use Archiva to retrieve dependencies of Maven Central from now on only from your Raspberry pi. I advise to not configure your Maven clients on your laptops, unless your Raspberry pi is accessible via the internet, or you have a VPN or something. Otherwise you won't be able to get your dependencies when your Raspberry pi is not on your current network.

Create or edit your Maven settings file from your user directory with your favorite editor,.

On modern Windows machines, this file should be located on:
c:\users\XXX\.m2\settings.xml (replace XXX wityh your username)

On Linux machines, this file is located at:
~/.m2/settings.xml

Make sure the setting.xml file contains at least something like: (but if you have other entries as well, like proxies, etc., make sure to retain them)

<?xml version="1.0" encoding="UTF-8"?>
<!--
    User-specific configuration for maven. Includes things that should not 
    be distributed with the pom.xml file, such as developer identity, along with 
    local settings, like proxy information. The default location for the
    settings file is ~/.m2/settings.xml 
-->
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
    <mirrors>
        <mirror>
            <id>archiva.default</id>
            <url>http://RASPBERRY_IP_ADDRESS:8080/archiva/repository/internal/</url>
            <mirrorOf>central</mirrorOf>
        </mirror>
    </mirrors>        
</settings>
    

Of course, replace RASPBERRY_IP_ADDRESS with your Raspberry pi's IP address.

The <mirrorOf>central</mirrorOf> entry tells Maven to only use your Raspberry pi mirror for Maven Central dependencies. Refer to the Maven documentation for more mirror configuration options. Also make sure to read the corresponding Archiva chapter on this subject.

Now when you build a project, Maven Central repositories will be downloaded from your Raspberry pi, which will automatically download the dependency if it did not have it already in its cache.

Some final thoughts

The (Micro-)SD card from your Raspberry pi can get corrupted when files change very often. So if you very frequently add new dependencies or change versions, it's probably better to attach a external harddrive to your Raspberry pi and make sure the Maven repositories are stored on that hard drive.

If after testing you don't want to use Archiva anymore, simply remove the <mirror>....</mirror> entries from your client's settings.xml and you should be fine.

To shutdown TomCat on your Raspberry, you can use standard ~/tomcat/bin/shutdown.sh script, but remember to start the server, use the script in your home directory, otherwise Archiva won't start because it can not find its home directory.

On my Raspberry pi, less than 250 MB of memory was used to run Linux, Tomcat and Archiva, so I have plenty of room to run other servlets on my Raspberry pi in the future.


Running Nashorn scripts in Ant build scripts: The basics

by vincent


Posted on Tuesday Dec 01, 2015 at 09:01AM in Build Tools


When I start a Java project that I think can confirm to the rules of Apache Maven, I use Maven to create build scripts. Nowadays this is in the majority of my web projects.

Occasionally though, there are cases where I want full control of every step and/or want to do a lot of exotic steps. In those (rare) cases, I still manually create Apache Ant XML build scripts. It has so many built-in type of tasks, most of which are easy to use. Unlike many other developers I know, I quite like Ant, especially when also using Apache Ivy for dependency management.

Checking out the new popular choice for building JVM projects, Gradle By Gradle, Inc., is another high entry on my ever -growing to-do list. Gotta love a tool that has a cursing elephant as a mascot, to illustrate the usual building frustrations ;-) . I will definitely check Gradle out soon.

As probably well known by now, I am a huge Oracle Nashorn fan. When I started working on my latest project, I figured it could be handy to run Nashorn scripts inside Ant scripts and found multiple solutions that I'd like to discuss. For now let's start with the most simple method.

Script Task

Nashorn is fully compatible with the JSR-223 (aka "Scripting for the Java Platform" standard), which Ant supports. Several enhancements were done in Ant 1.7 to this task, so I assume you use a somewhat recent Ant version.

To simply run Nashorn scripts as part of a Ant target, you can use the <script> task. You can embed the script directly in the build.xml file (yuck!) or point it to an external file containing the script.

A simple build.xml example that embeds the script in the XML file (something that I'd never do in production!)

<?xml version="1.0" encoding="utf-8"?>
<project name="VincentsProject" basedir="." default="main">

	<property name="message" value="Hello!"/>
	
	<target name="main">
		<script language="nashorn" > <![CDATA[
			print(message);
			print(project.getProperty("message"));
			print;
			print(project.name);
			print(VincentsProject.name);
			print;
			project.log("Logged from script", project.MSG_INFO);			
		]]></script>
	</target>
</project>
    

You can run the script by saving above's text in a "build.xml" file, switch to a terminal window (Command Prompt on Windows) and run "ant". The output should be something like:

Console output example

Consult Ant's Script task documentation and note some things here:

  • It would have been much, much better to store the script in an external file in a subdirectory and add a src="./build_scripts/script1.js" alike attribute to your <script> attribute.

  • Ant makes available all defined properties to the external script. That's why
    print(message), print(project.getProperty("message")) and print(VincentsProject.name)
    lines all work. You can disable this behavior by adding the setbeans=false attribute to the <script> element, in that case only the "project" and " self" variables would have been passed to the script and "VincentsProject" and "message" would not have been available.

  • You could use the language="javascript" attribute in the <script> element, but this would run Rhino on Java versions 1.6 and 1.7. It is very likely your Nashorn JavaScript script will not be compatible with Rhino, so I'd recommend to use language="nashorn" attribute, unless you are sure your script is compatible with both, or your project requires Java 1.8 or later anyway.

  • Don't worry about the "manager" attribute. Its default value "auto" should be fine. Or set it to "javax" if you're a purist. From what I understand, Apache BSF was an older scripting standard that predates and inspired the JavaX JSR-223 specification that Nashorn implements.

  • When your Nashorn script uses external Java libraries, you can add a "<classpath>" element to the <script> element, like: <classpath><fileset dir="lib" includes="*.jar" /></classpath>

  • You can also add a classpath by defining a "<path>" element in your build file under "<project>" element; and refer to this inside your script element by using the "classref" attribute. See Ant documentation for more help. Use this construction if you need the same classpath specification for multiple tasks.

  • One thing that not seems possible is to add parameters to the ScriptFactory. AFAIK it's therefore not possible to use Nashorn's shell scripting capabilities when using this Script task.

Note that you can do some funky stuff. You can implement a full Ant task that takes file sets as a parameter. I'll rewrite one of the examples from the Ant documentation in Nashorn and post it here.

In a next post about this subject, I will post about the Scriptdef task, that adds some interesting features.