Create WebSphere Liberty Server Offline Buildpack

A number times I get queries on how to create an offline version of the WebSphere Liberty Profile Buildpack. Here are the high level steps with some color added, where the instructions are not clear for creating an Offline version of the liberty buildpack. 

The instructions below assume access to the internet. Once the buildpack is created then you can push applications to it. The compile process of the buildpack will find all the dependencies within the buildpack.

The steps below should be automated in a CI pipeline. For an example of a CI pipeline that generates the liberty offline buildpack head over to travis.

## Instructions Loosely Followed:

If you have customization to WebSphere  and JRE binaries.
- [Forking The Buildpack](https://github.com/cloudfoundry/ibm-websphere-liberty-buildpack/blob/master/docs/forking.md)

If you have NO customization to WebSphere and JRE binaries.
- [How to install the Buildpack into your Cloud Foundry release](https://github.com/cloudfoundry/ibm-websphere-liberty-buildpack/blob/master/docs/installation.md)

## Download WAS and JRE Binaries
https://developer.ibm.com/assets/wasdev/#asset/runtimes-wlp-runtime wlp runtime jar and others needed needed to setup repository see below ...

https://www-01.ibm.com/marketing/iwm/iwm/web/preLogin.do?source=swg-sdk8&S_PKG=amd64_8.0.3.0&S_TACT=105AGX05&S_CMP=JDK
- IBM 64-bit SDK for Linux on AMD64/EM64T architecture, Java Technology Edition
  Version  8.0.3.0
- Simple unzip with license (InstallAnywhere root not required)
  ibm-java-jre-8.0-3.0-x86_64-archive.bin  (111 MB)


~/Downloads/liberty8559❯
-rw-r--r--@   1 kelapr  720748206  147038277 Apr 30 00:06 ibm-java-x86_64-sdk-8.0-3.0.bin
-rw-r--r--@   1 kelapr  720748206   62753431 Apr 30 00:00 wlp-runtime-8.5.5.9.jar

## Structure the binaries into a repository and create the appropriate index.yml files 

https://github.com/cloudfoundry/ibm-websphere-liberty-buildpack/blob/master/docs/util-repositories.md#setting-up-your-web-server

Copy license files from
https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/wasdev/downloads/wlp/8.5.5.9/lafiles/runtime/en.html
https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/wasdev/downloads/jre/index.yml

Push these binaries to the cloud using the static file buildpack
Dir. Structure before push...

~/Downloads/liberty8559❯ tree -f .
.
├── ./Staticfile
└── ./buildpack
    ├── ./buildpack/jre
    │   ├── ./buildpack/jre/ibm-java-jre-8.0-3.0-x86_64-archive-License.html
    │   ├── ./buildpack/jre/ibm-java-jre-8.0-3.0-x86_64-archive.bin
    │   └── ./buildpack/jre/index.yml
    └── ./buildpack/wlp
        ├── ./buildpack/wlp/index.yml
        ├── ./buildpack/wlp/wlp-developers-runtime-8.5.5.9-License.html
        ├── ./buildpack/wlp/wlp-javaee7-8.5.5.9.zip
        ├── ./buildpack/wlp/wlp-kernel-8.5.5.9.zip
        ├── ./buildpack/wlp/wlp-runtime-8.5.5.9.jar
        └── ./buildpack/wlp/wlp-webProfile7-8.5.5.9.zip



Modify the index.yml files after pushing for the first time and then repush

~/Downloads/liberty8559❯ cf push liberty-repo
Updating app liberty-repo in org pivot-rkelapure / space development as rkelapure@pivotal.io...
OK

See http://liberty-repo.cfapps.pez.pivotal.io/ for a functioning repository and index.yml files for a minimal repo of customized liberty and jre.

Another way of creating a repository is to run the dependency-builder; however configuring and running the dependency builder takes time and is error prone. Your best chance of success is to handcraft the repository based on your custom websphere binaries. If you have NO access to the internet when running the package rake task THEN you will need to setup a repository with not only WAS and JRE binaries but also all the third party binaries that are used like dynatrace, appdynamics etc.,

Modify the code in ibm-websphere-liberty-buildpack/config/ibmjdk.yml to point to the pushed JRE.
Modify the code in ibm-websphere-liberty-buildpack/config/liberty.yml to point to pushed Liberty.

Create the offline buildpack
https://github.com/cloudfoundry/ibm-websphere-liberty-buildpack/blob/master/docs/installation.md
Create a licenses.yml based on the license files uploaded above

It is critical that the repository index.yml (specifically the wlp/index.yml)  are setup right otherwise you will see errors like these...

rake aborted!
undefined method `gsub' for nil:NilClass
/Users/kelapr/git/ibm-websphere-liberty-buildpack/resources/download_buildpack_cache.rb:155:in `filename'
/Users/kelapr/git/ibm-websphere-liberty-buildpack/resources/download_buildpack_cache.rb:64:in `block in download_cache'
/Users/kelapr/git/ibm-websphere-liberty-buildpack/resources/download_buildpack_cache.rb:47:in `each'
/Users/kelapr/git/ibm-websphere-liberty-buildpack/resources/download_buildpack_cache.rb:47:in `download_cache'
/Users/kelapr/git/ibm-websphere-liberty-buildpack/Rakefile:95:in `block (2 levels) in <top (required)>'
/Users/kelapr/git/ibm-websphere-liberty-buildpack/Rakefile:78:in `block in <top (required)>'
/Users/kelapr/.rvm/gems/ruby-1.9.3-p545/bin/ruby_executable_hooks:15:in `eval'
/Users/kelapr/.rvm/gems/ruby-1.9.3-p545/bin/ruby_executable_hooks:15:in `<main>'

## Generate the Buildpack

kelapr @ usxxkelaprm2 in ~/git/ibm-websphere-liberty-buildpack on git:master x [18:31:11]
$ bundle exec rake 'package[,*,]'

This will generate the offline IBM WebSphere Buildpack one dir. above
ibm-websphere-liberty-buildpack.zip

You can confirm the output location from the following line
/Users/kelapr/git/ibm-websphere-liberty-buildpack.zip as a buildpack zip output file

Once the buildpack is created. you will find all the cached dependencies inside

$ unzip -vl ibm-websphere-liberty-buildpack.zip

Archive:  ibm-websphere-liberty-buildpack.zip
 Length   Method    Size  Ratio   Date   Time   CRC-32    Name
--------  ------  ------- -----   ----   ----   ------    ----
       0  Stored        0   0%  05-01-16 18:26  00000000  admin_cache/
31318727  Defl:N 30838619   2%  05-01-16 18:24  7d5deee0  admin_cache/http%3A%2F%2Fdownloads.dynatracesaas.com%2F6.3%2Fdynatrace-agent-unix.jar.cached
      82  Defl:N       76   7%  05-01-16 18:24  edd887d4  admin_cache/http%3A%2F%2Fdownloads.dynatracesaas.com%2Fcloudfoundry%2Fbuildpack%2Fjava%2Findex.yml.cached
   71744  Defl:N    20247  72%  05-01-16 18:24  e514cbb9  admin_cache/http%3A%2F%2Fliberty-repo.cfapps.pez.pivotal.io%2Fbuildpack%2Fjre%2Fibm-java-jre-8.0-3.0-x86_64-archive-License.html.cached
115512196  Defl:N 114652626   1%  05-01-16 18:25  7b5cfed8  admin_cache/http%3A%2F%2Fliberty-repo.cfapps.pez.pivotal.io%2Fbuildpack%2Fjre%2Fibm-java-jre-8.0-3.0-x86_64-archive.bin.cached
     234  Defl:N      124  47%  05-01-16 18:24  f5c67578  admin_cache/http%3A%2F%2Fliberty-repo.cfapps.pez.pivotal.io%2Fbuildpack%2Fjre%2Findex.yml.cached
     596  Defl:N      171  71%  05-01-16 18:25  f94f0939  admin_cache/http%3A%2F%2Fliberty-repo.cfapps.pez.pivotal.io%2Fbuildpack%2Fwlp%2Findex.yml.cached
   60989  Defl:N    18625  70%  05-01-16 18:25  44573b38  admin_cache/http%3A%2F%2Fliberty-repo.cfapps.pez.pivotal.io%2Fbuildpack%2Fwlp%2Fwlp-developers-runtime-8.5.5.9-License.html.cached
67959662  Defl:N 67748071   0%  05-01-16 18:26  588faaec  admin_cache/http%3A%2F%2Fliberty-repo.cfapps.pez.pivotal.io%2Fbuildpack%2Fwlp%2Fwlp-webProfile7-8.5.5.9.zip.cached
    1487  Defl:N      183  88%  05-01-16 18:25  f74bed4e  admin_cache/https%3A%2F%2Fdl.zeroturnaround.com%2Fjrebel%2Findex.yml.cached
27633836  Defl:N 25152183   9%  05-01-16 18:25  04291789  admin_cache/https%3A%2F%2Fdl.zeroturnaround.com%2Fjrebel%2Freleases%2Fjrebel-6.4.3-nosetup.zip.cached
11656197  Defl:N 11640688   0%  05-01-16 18:24  a9417282  admin_cache/https%3A%2F%2Fdownload.run.pivotal.io%2Fapp-dynamics%2Fapp-dynamics-4.1.8_5.zip.cached

....

## Update PCF with Buildpack and push application with custom buildpack

# kelapr @ usxxkelaprm2 in ~/git [19:11:47]
$ cf create-buildpack  rliberty ./ibm-websphere-liberty-buildpack.zip 10 --enable

cf push spring-music -p build/libs/spring-music.war -b rliberty