The JVM Fanboy Blog 

Host Maven repositories on a Raspberry Pi

by vincent


Posted on Sunday May 08, 2016 at 11:42PM in General


For a particular project I'm doing, it would be handy to re-use a JAR file in multiple other future projects. Although there are far more simple solutions to this problem, when I read about repository managers in the official Maven documentation, I gave it some thought and decided to try to run one on one of my Raspberry pi mini-computers that was waiting for a problem to be solved :-)

When hosting a Maven repository on the Raspberry pi, next to hosting private repositories that can be accessed by other computers in my network, it can act as a proxy for Maven Central as well. This means that when a computer in my network needs to download a dependency from Maven Central, it will get a cached copy from the Raspberry pi instead. If the dependency is not already in its cache, it will download the dependency from Maven Central (if it is configured to allow this) first.

Apache Archiva

Looking at the list of available Maven Repository Managers, Apache Archiva caught my eye and decided to give it a try.

Like many other Apache Software Foundation projects that I follow, Archiva is a project with light traffic and not a lot of new releases lately, this is not necessarily a bad thing though, hopefully this is a sign that the current version is generally considered stable.<.p>

What I saw when looking up for more information, impressed me. The web user-interface looks really user-friendly and nice.

Browsing repository in Archiva

Running Archiva in Standalone mode or Servlet container?

Archiva can run in two modes:

  • Stand-alone (a script can be started on the console, that will start an embedded web-server)
  • Run inside a Servlet container (Apache Tomcat) or dedicated Java EE application server (Glassfish, JBoss/Wildfly, Oracle WebLogic...)

Let's take a look at both options

Standalone mode on the Raspberry pi

At first I tried running Archiva in stand-alone mode, assuming this would be ideal for the Raspberry pi.

I had many issues along the way, which all had to do with Archiva's dependency of an out-dated version of Java Service Wrapper by Tanuki Software. As I understand Service Wrapper is a component that lets Java applciations run as Linux daemons and Windows services. Due to licensing issues, the Archiva team can not upgrade this component to a more up to date version.

Blogger Ti-80 - almost exactly three years ago - made a blog post about running Archiva on the Raspberry pi and it seems all problems can be solved by manually building the project and replacing binaries.

I decided to not follow this route and run it in a servlet container. I reasoned I'll probably want to run additional servlets on this Raspberry pi in the future anyway. And Adam Bien convinced me that Tomcat's memoy consumption is not as excessive as some people seem to think.

Running inside Servlet Container

Here are the steps that I took to get this up and running. I did not follow the documentation exactly, as I have some other conventions. Feel free to disagree with me and use their instructions! Note that I use a Raspberry pi 2 (4 cores and 1GB of internal memory), I have not tried this on other models.

The official installation guide is here.

  • Login with SSH to your Raspberry pi, I used the built-in pi user (you did change its default password, didn't you?! :) )
  • Create a dedicated Linux user for running Tomcat:

    sudo -s
    adduser tomcat

    Follow the prompts, then change active user to "tomcat"

    su tomcat

  • Let's create directories and download Tomcat and Archiva

    cd ~
    mkdir downloads
    cd downloads

    Visit the TomCat homepage and look for the download link of the .tar.gz release of the currently stable version. At the time of this writing this was 8.0.33

    wget URL-TO-TOMCAT.tar.gz
    (replace "URL-TO-TOMCAT.tar.gz" with the download URL)
    tar xvfz ./apache-tomcat-X.Y.Z.tar.gz
    (replace "apache-tomcat-X.Y.Z.tar.gz" with downloaded file)
    mv ./apache-tomcat-X.Y.Z ~/tomcat
    (move the created directory, not the downloaded tar.gz file!, to the home directory)

  • Visit the Archiva homepage and look for the download link of the WAR release of the currently stable version. At the time of this writing this was 2.2.0

    wget URL-TO-ARCHIVA.war (replace "URL-TO-ARCHIVA.war" with the download URL)

  • I create a dedicated directory for Archiva and do not place it in the Tomcat home directory (which the Installation Guide suggests). I don't want to pollute Tomcat home directory with Archiva related files. This is a debatable decision, as some dedicated TomCat configuration files will be required to run Archiva anyway.

    cd ~
    mkdir -p webapps/archiva
    cd webapps/archiva
    mkdir conf
    mkdir db
    mkdir logs
    mkdir war

    cd war
    cp ~/downloads/*archiva* ./

  • Let's create Tomcat context configuration XML file.

    cd ~/tomcat/conf
    mkdir -p Catalina/localhost
    cd Catalina/localhost
    nano archiva.xml
    (Install nano if it could not be found: apt-get install nano)

  • Enter following code in Nano
    (Substitute the docBase path with the full path to your downloaded war file)

    <?xml version="1.0" encoding="UTF-8"?>
     <Context path="/archiva"
              docBase="/home/tomcat/webapps/archiva/war/apache-archiva-2.2.0.war">
    
     <Resource name="jdbc/users" auth="Container" type="javax.sql.DataSource"
               username="sa"
               password=""
               driverClassName="org.apache.derby.jdbc.EmbeddedDriver"
               url="jdbc:derby:/home/tomcat/webapps/archiva/db/users;create=true" />
    
     <Resource name="mail/Session" auth="Container"
                type="javax.mail.Session"
                mail.smtp.host="localhost"/>
     </Context>
                

    Press CTRL+X , then Y to exit.

  • A bit annoyingly, you'll need to install some dependencies in the "lib" directory of Tomcat

    You could download the Archiva .zip release and get it from them, but you could also download the files from mvnrepository.com, what we'll do here. I've chosen the exact versions used by the current Archiva stable version to prevent conflicts. We could regret this later, when installing applications that require newer versions... :( .

    Check the installation guide to see if the version numbers mentioned here still match with the latest version!

    cd ~/tomcat/lib

    Vist http://central.maven.org/maven2/javax/activation/activation/1.1/activation-1.1.jar and find the "Download JAR" button. Copy the link URL.

    wget LINK-TO-ACTIVATION-1-1.JAR (replace with full link to activation-1-1.jar)

    Vist http://mvnrepository.com/artifact/javax.mail/mail/1.4 and find the "Download JAR" button. Copy the link URL.

    wget LINK-TO-MAIL-1-4.JAR (replace with full link to mail-1.4.jar)

    Vist http://mvnrepository.com/artifact/org.apache.derby/derby/10.12.1.1 and find the "Download JAR" button. Copy the link URL (manual states newer versions of Derby than mentioned in the documentation should work fine).

    wget LINK-TO-DERBY.JAR (replace with full link to Derby jar file)

  • Finally create a start script that will boot Tomcat and additionally sets some required environmental variables.

    cd ~
    nano startup.sh

    Add the following content:

    export CATALINA_OPTS="-Dappserver.home=/home/tomcat/webapps/archiva -Dappserver.base=/home/tomcat/webapps/archiva"
    /home/tomcat/tomcat/bin/startup.sh
             

    Press CTRL+X , then Y to exit.

    chmod +x ./startup.sh

  • Run the script:
    ./startup.sh
  • On the computer you used to log in to Raspberry pi, start your browser (so do not run the browser on your Raspberry pi), go to:
    http://IP_ADDRESS_RASPBERRY_PI:8080/archiva
    (replace IP_ADDRESS_RASPBERRY_PI with the correct ip address or host name)

    If everything goes well, after a few seconds you should see a welcome screen, with a button at the top right side to create an admin user. Note that booting can take some time.

    On problems, you'd get a simple 404 screen. In that case you'll have to look at the log files of Tomcat and try to determine what is wrong, usually something's wrong with one of the paths or dependencies:
    nano /home/tomcat/tomcat/logs/localhost.YYYY-MM-DD.log (replace with your current date)

  • Create the admin user and follow the prompts.
  • I should do a future blog post about configuring Archiva, I feel the default settings are good enough to get started.

Configure Maven clients to use Archiva

To get started, let's configure your desktop machine clients to use Archiva to retrieve dependencies of Maven Central from now on only from your Raspberry pi. I advise to not configure your Maven clients on your laptops, unless your Raspberry pi is accessible via the internet, or you have a VPN or something. Otherwise you won't be able to get your dependencies when your Raspberry pi is not on your current network.

Create or edit your Maven settings file from your user directory with your favorite editor,.

On modern Windows machines, this file should be located on:
c:\users\XXX\.m2\settings.xml (replace XXX wityh your username)

On Linux machines, this file is located at:
~/.m2/settings.xml

Make sure the setting.xml file contains at least something like: (but if you have other entries as well, like proxies, etc., make sure to retain them)

<?xml version="1.0" encoding="UTF-8"?>
<!--
    User-specific configuration for maven. Includes things that should not 
    be distributed with the pom.xml file, such as developer identity, along with 
    local settings, like proxy information. The default location for the
    settings file is ~/.m2/settings.xml 
-->
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
    <mirrors>
        <mirror>
            <id>archiva.default</id>
            <url>http://RASPBERRY_IP_ADDRESS:8080/archiva/repository/internal/</url>
            <mirrorOf>central</mirrorOf>
        </mirror>
    </mirrors>        
</settings>
    

Of course, replace RASPBERRY_IP_ADDRESS with your Raspberry pi's IP address.

The <mirrorOf>central</mirrorOf> entry tells Maven to only use your Raspberry pi mirror for Maven Central dependencies. Refer to the Maven documentation for more mirror configuration options. Also make sure to read the corresponding Archiva chapter on this subject.

Now when you build a project, Maven Central repositories will be downloaded from your Raspberry pi, which will automatically download the dependency if it did not have it already in its cache.

Some final thoughts

The (Micro-)SD card from your Raspberry pi can get corrupted when files change very often. So if you very frequently add new dependencies or change versions, it's probably better to attach a external harddrive to your Raspberry pi and make sure the Maven repositories are stored on that hard drive.

If after testing you don't want to use Archiva anymore, simply remove the <mirror>....</mirror> entries from your client's settings.xml and you should be fine.

To shutdown TomCat on your Raspberry, you can use standard ~/tomcat/bin/shutdown.sh script, but remember to start the server, use the script in your home directory, otherwise Archiva won't start because it can not find its home directory.

On my Raspberry pi, less than 250 MB of memory was used to run Linux, Tomcat and Archiva, so I have plenty of room to run other servlets on my Raspberry pi in the future.


Why the JVM is my favorite platform

by vincent


Posted on Wednesday Nov 18, 2015 at 12:27AM in General


There's currently a lot being written about the somewhat uncertain future of Java. Actually, there's so much being written about this (be sure to click on that last link), that I don't have much to add to this discussion.

Still, I'd like to describe here why the Java platform is my favorite development platform and I will keep on using it on my next projects for the foreseeable future.

Building everything in only one language is a logical, but not necessarily the best choice.

I have changed jobs a few times over the last decade and each one required me to learn a new language.

Having worked with both static and dynamic languages (learning a pure functional language is high on my wish-list) for several years, on different platforms (Windows and Linux mostly), I see advantages and disadvantages for each language and platform choice.

More and more companies permit their development teams to use the tool that the team feel is right for the job. Well known examples of companies that let their team use different languages for different projects are Netflix, LinkedIn, Spotify, Last.FM, etc. I welcome this change and hope more companies will follow eventually.

In my country, The Netherlands, I don't really see this shift happening at the moment, but my guess is it will be the future of development.

JVM: mix and match languages, libraries and even operating systems

The JVM makes it easier (and even fun) to use different languages/operating systems, and still be compatible with many libraries and other development tools.

To me, this is one of the highlights of the platform. Embedding Python script languages in pure Java projects thanks to Jython, using pure Java libraries in the JavaScript language with Nashorn, or writing compact unit-tests code in Groovy for my Java code is a very fun aspect and, in my experience, boosts productivity a lot.

Personally, I develop many of my projects on a Windows machine and deploy the very same JAR file, containing all dependencies that I have tested on my machine, on a Linux server. Here Java developers have more freedom as well than users of most other languages.

Another advantage of having multiple languages is this: I'm sure many new Java 8 features were inspired by the dynamic JVM Groovy language. Having more languages from different vendors/teams on one platform is a very positive thing, it keeps everyone awake.

Modern feature-set

Other highlights:

  • Java is focused on performance. Being able to run threads on different CPU cores is very important to me. We have multi-core CPUs for a very good reason.
  • Thanks to Java 8's Streams API, it's never been easier to make use of multiple cores with minimal code changes. It must be mentioned Java 7 added great multi-threading features as well.
  • JVM developers have so much choice: application servers, databases (even natively written in Java), enterprise application frameworks, mature web frameworks, ORM, etc.
  • Very good development tools. Again a lot of choice, like NetBeans, IntelliJ, Eclipse, JDeveloper IDEs.
  • Memory consumption and boot time can still be a problem, but Java 9 is expected to improve this as part of their modularity Jigsaw project

Do something back...

I think there's something that we Java developers can do to try to keep the Java community healthier. If your company is making money and running on many different open-source projects, consider donating back to at least one of those projects (create patches to fix bugs, improve documentation....). In the very least blog actively about your usage of those projects.

If your company can afford it, instead of automatically using the open-source GlassFish enterprise application server, consider reviewing the commercial Oracle WebLogic server. Oracle is a commercial company and probably wouldn't mind making money on its Java products.

It's probably wishful thinking on my part, I know it's much easier said than done.

Do not forget that the JVM is also not the right tool for everything

While my blog is called JVM Fanboy, I have a lot of interests in other programming languages as well. As a developer you should never put all eggs in one basket.

Node.JS for example is simply incredible. Also I work with Python daily and enjoy that a lot as well. Those languages get more than enough press time, so that's why I decided to blog about JVM-related stuff exclusively. But that does not mean that those other popular languages don't deserve their popularity.


Welcome!

by vincent


Posted on Sunday Jul 05, 2015 at 03:28PM in General


Hello!

Welcome to my blog dedicated to anything that is - somehow :) - related to the Java Virtual Machine.

Java and the JVM have come a long way, a lot of the well-known prejudices are simply not true anymore, or are actively being taken care of. Don't believe me? Ask Twitter.

I love that these days the JVM is a host for many other programming languages and applications can make use of classes without caring which JVM programming language was used to compile that class file. Also more and more dynamic languages are appearing that do not require compiling by the user, but still can fully utilize Java classes. Often, those languages can easily be embedded in Java projects as well.

I follow development of and news reports about Java, the JVM, alternative languages, developer tools, libraries, frameworks quite a bit, so I think I'll have quite a lot to blog about.