Migrating WebSphere Application Server Applications to Cloud Foundry

Migrating WebSphere Application Server Applications to Cloud Foundry

Migration of WebSphere Application Server apps to Cloud Foundry entails dissecting the app for API usage and then configuring the implementation of the same APIs on a servlet container or an alternate open source EE server. My app server of choice for Java EE APIs is Tomcat + Spring Framework or Apache TomEE. 

Migrations of Java EE applications vary in complexity, depending on a series of factors that are identified during the migration:

• Source and target Java SE specification (JRE 1.5 to JRE 1.8)
• Source and target Java EE specifications (J2EE 1.4 to Java EE 7)
• Usage of specific Java components (EJB, JSPs, JSF, etc.,)
• Usage of proprietary code that is specific for an application server
• Vendor-specific deployment descriptors
• Extended capabilities
• Vendor-specific server configurations (JMS providers and JDBC connection pooling)
• Class loading issues .
• Integration with complex external systems such as a CRM or SAP

From a code perspective an application owner has to ask questions related to Java, EJB usage, Servlets & JSPs, Web Services, database access, JMS, JNDI naming, Application trace & logging, UI frameworks, Transactions, Threads, sockets and XML/REST/JSON. The configuration and usage of these technologies will affect the migration of the app to Cloud Foundry. From a deployment perspective the following areas need to be probed: Current hardware, current network edge load balancing solution and expectation around session affinity, expected downtime during upgrade, current administration constructs, Details of the HTTP Server/Proxy Tier. hardware details & security (How exactly is the security of the app and app-server managed). From an architectural perspective to understand the technical barriers and issues in porting apps to cloud native CF platform please read Roadblocks to PaaS.

Process

1. Get a lay of the land. Identify ALL the open source and proprietary APIs used by the app. Determine what modifications you will need to make to the app if the target runtime packaged by the buildpack does not include the said libraries.

   APIs

   • Make a list of all WebSphere Proprietary APIs used by your app,
   • Make a list of all the Java EE APIs used by your app,
   • Make a list of all Spring Framework APIs used the app,
   • Make a list of all 3rd party open source frameworks used by the app,

   Analysis Tools

   • Leverage static analysis tools like FindBugs, Checkstyle, PMD, Coverity and SonarQube. Write custom rules that will provide a breakdown of the external api usage. From the The Wise Developers' Guide to Static Code Analysis - PMD is a very customizable tool, so the performance and installation outcome really depends on how one has instructed the tool to find problems. Coverity is a commercial tool that it does a much deeper analysis than other tools and it also keeps track of the issues that you’ve found and fixed. SonarQube gives the best combination of usefulness and transparency against the relative complexity of setting it up and solving weird configuration issues.
   • Run Google code search on your app source repository. Code Search is a tool for indexing and then performing regular expression searches over large bodies of source code. It is a set of command-line programs written in Go.
   • Use a modern IDE like Netbeans, Eclipse and IntelliJ to find usage and references of classes and packages in your app code.
2. Look at all the XML files bundled within the app. These XML Deployment descriptors can be categorized into two categories. Standard discriptors and app-server specific ones. You will need to replace the vendor specific depkoyment descriptors with TomEE descriptors or Spring Beans in Tomcat.
   
   
   
3. If possible disaggregate the EAR file into multiple war files and tie them together as one logical app. This will increase loose coupling and force you to disentangle a web of dependencies that will make the transition to a microservices architecture much easier. You can chose to lift and shift your ear file as-is too since some of the buildpacks below will allow the staging of ear files. Typically it is OK to duplicate code like utility jars across multiple war files it leads to loose coupling and better isolation of function. Note as of today only the Liberty buildpack supports the cf push of ear files. The Tomcat based Java buildpack does not support ear files.
   
4. Look at shared libraries configured for the app server across apps. These server scoped shared libraries will need to be bundled in the app when it is pushed to CF.
   
5. Take advantage of Java EE6 and collapse EJBs and Servlets in a single war file. No classloader boundaries between Servlets and EJBs. EJBs and Servlets can share all third-party libraries (like Spring!), Use annotations and free your code from deployment descriptor hell.
   
6. Do not concern yourself with anything that deals with clustering, workload management, high availability, smart routing or proxying. These concerns are now taken care of by the underlying platform i.e. CF.
   
7. Check if there is any native code and ensure that the native library is included with the app and built on the linuxfs32 file system. When porting across Java EE versions please consider the breaking changes across EE specifications detailed in 3.3 J2EE to Java EE migration considerations
   
8. Address runtime Migration Concerns:
   • A common issue in migrating JMS resources is that the way each application server configures JMS resources varies from vendor to vendor. Before migrating JMS definition to WebSphere, you should decide which messaging engine to use to exchange messages asynchronously. CF provides multiple JMS & AMQP messaging providers.
   • Reconfiguring the JDBC resources is a common migration step. Use the appropriate JDBC driver to avoid problems with new implementations for new databases. Java Buildpack makes it easy to consume SQL and NoSQL data stores in the app running on the platform. You can auto-configure connections to back-end services or use spring-cloud-connectors to explictly control the JDBC connection to the persistence tier in the cloud.
   • There are several areas where you configure security resources for your migrated applications and the administrative functions and roles, such as authentication mechanisms, authorization mechanisms, and security for web services. Configure the target app server in Cloud Foundry with the correct AAA mechanisms to port all your security apparatus. Apache Tomcat 8 Security Considerations
   • Port JVM arguments, class path, and system properties to the buildpack and cf push manifests in Cloud Foundry.
   • Understand the external factors that affect migration like interaction with external CRM/ERP systems via XML or legacy web-service stacks.
9. By default, the Tomcat instance provided by the Java Buildpack is configured to store all sessions and their data in memory. If your app relies on session persistence then configure session replication in CF by binding the app to a DB like redis session replication on Cloud Foundry. In some cases the configuration of the app to use the session database may need to be done explicitly based on the service variables injected via the VCAP_SERVICES environment variable. CF by default supports sticky session i.e. honors the JSESSIONID cookie.
   
10. If the app does not run on WebSphere Liberty Profile then use the WebSphere Migration Toolkit to get the app running on the WebSphere Liberty Profile. The Liberty Tech Preview tool helps you move WebSphere Application Server full profile applications to Liberty profile, which can be running inside or outside of the cloud. Follow the steps outlined by Cindy High on moving applications to Liberty using the migration toolkit.
    
11. You have now come to a fork in the road. You have multiple choices [AppServer(WebSphere | TomEE | JBOSS) * Cloud packager(Buildpack | Docker)] in deploying your app to CF.
    
    1. Run the app in Cloud Foundry using the WebSphere Liberty Profile Buildpack. WebSphere Liberty Profile provides production support for Java EE7 and requires zero-migration across versions of the app server. The licensing model of running Liberty in production are NOT clear. The way I read the license, is that WebSphere Liberty profile can be used in production on CF for JVM heaps <= 2GB. So this gives you the license to install Liberty and fixes thereafter; not entitlement to support. No serious enterprise can run all their business on 2GB JVM heap. The catch here is that all your apps in total across the org cannot exceed 2 GB in heap. You can share 2GB across as many instances of Liberty as you like as long as the total JVM heap space across the instances does not exceed 2GB. ;) . Basically the first instance is free and thereafter you pay. Of course this licensing does not apply to Bluemix where you can run as many instances as you like as long as you pay Bluemix GB/hr cost. 
    2. Run the app in Cloud Foundry using TomEE. See this blog post from Romain Manni-Bucau that explains how to get the app running on Bluemix which is the IBM derivative of CF. The same technique will work on open source and other distributions of CF. This technique stages and runs the app as an embedded TomEE server within the app. 50 Shades of TomEE explains the many ways an executable fat jar can be cooked with your app and TomEE binaries. Another popular packaging option is to follow recipes and prepare a docker container of the app using the official TomEEDocker image from TomEETribe.  Deployment of Docker image in Cloud Foundry is possible with the Diego BOSH release. Diego supports running of Docker images natively within CF.  The recommendation here is to ONLY package the app in Docker and rely on the platform to inject service metadata. A great guide on packaging apps with Docker and keeping your docker images can be found in Adam Bien's blog JAVA EE 7 + THIN WARS + DOCKER = GREAT PRODUCTIVITY.
    3. Stage and run the app in JBoss now renamed as WildFly, JavaEE certified application server using the jboss-buildpack. Redhat has provided a detailed guide on How-to migrate application running on WebSphere to WildFly was2jboss. If your application is not cloud-native then package and deploy your app using the official WildFly Docker image. 
    4. Refactor the app based on decomposition recipes outlined by Matt Stine in Migrating to Cloud-Native Application Architectures such as the anti-corruption layer and strangling the monolith. As you refactor the app to a microservices based architecture use SpringCloud OSS to implement the distributed services patterns necessary to overcome the fallacies of distributed computing. A playbook on app refactoring is provided by Josh Kruck here. Run the refactored slmmed down app on a specialized micro-server like Spring-Boot or DropWizard, What we want is a simple, lightweight, fire-and-forget packaging tool for server-side code taking advantage of executable JARs’ simplicity and platform independence. Merging all dependencies into a single JAR can be done with Tools like Maven’s Shade plugin or Gradle’s Shadow plugin. Before deciding on Spring Boot or DropWizard understand the tradeoffs as explained by the good folk at Takipi.
    5. If you do not want to put any effort in refactoring the app since it is of low strategic value and business impact then simply packing the app in a Docker container and pushing it to PCF will allow you to deploy the app to the platform and leverage the operational & cost benefits with very little development cost. Run the IBM WebSphere Application Server Full and Liberty profiles in a Docker Container following documentation in FullWasInDocker and LibertyInDocker. Again Licensing headaches apply :-(. To get started with running in WebSphere go through this IBM InterConnect 2015 session on WebSphere Application Server Liberty Profile and Docker.

IBM Bluemix & Pivotal Cloud Foundry - A Tale of Two PaaS

Parameters for comparison
- Ramp for Developers
- Devops Experience
- Pushing Docker Images
- Services (Beta and Production)
- Cost for a stress test
- 

PaaS and the Disruption of the Financial Tech Industry

A number of companies from the financial services industry are looking at Cloud Foundry or already have production deployment of Cloud Foundry. Why is the deployment  and uptake of a platform, particularly strong in Financial Tech ?  The reason is that this is an industry that is in a fervor of disruption and disaggregation. The incumbents have taken notice and are coming to the battle with guns drawn. Take a look at this picture.

http://fermi.vc/post/72559525330/disaggregation-of-a-bank

As we look at the future of  enterprises with CF, as senior consultants and technical leaders it behooves us to paint a picture of the future. For a look at the ideal digital future of this kind of platform driven transformation you need to look no further than the European bank ING who is undergoing/underwent a complete digital transformation. ING CEO Koos Timmermans spoke at the Goldman Sachs European Financials Conference in Rome on June 16/2015. Take a look at this slide deckhttp://www.slideshare.net/ING/goldman-sachs-16-june-2015 in particular slides 6 and 9. Note ING used a home-grown PaaS and not CF. To quote Koos from the slide deck - 

Investment of EUR 200 mln to further simplify, standardize and automate IT: • Decommissioning 40% of application landscape • Moving 80% of applications to zero-touch private cloud.

I also really enjoyed the following four part series from ThoughtWorks where the authors share their experiences and insights on ushering tech. fueled innovation in incumbent financial services organizations.

1. http://www.thoughtworks.com/insights/blog/innovation-eating-consumer-finance
2. http://www.thoughtworks.com/insights/blog/great-financial-services-companies-deliver-products-not-projects
3. http://www.thoughtworks.com/insights/blog/winning-companies-master-strategic-technology
4. http://www.thoughtworks.com/insights/blog/how-practice-continuous-innovation
5. http://radar.oreilly.com/2015/05/finance-transformation-oreilly-next-money.html

Much of what these articles state we already know; however the information is provided in the context of the financial tech industry and peppered with customer names/ case studies. These articles propose looking at all IT through a bi-modal view - Strategic and Utility IT. Thereafter they propose running strategic IT via a LEAN methodology in an innovation lab

• Shift from a project mentality to a product paradigm
• Break down large-scale, high-risk product development efforts into smaller, low-risk experiments.
• Provide these teams with the budget discretion to quickly test multiple ideas in parallel.Teams have toc learly define the problem and identify appropriate metrics to measure success.
• Collaborate early and often with stakeholders from technology, business, marketing, advertising, product, etc.

Technology Barriers To Moving Apps and Services to a PaaS

Technology Barriers To Moving Apps and Services to a PaaS

Language/Runtime Characteristics: CF only supports languages that have a buildpack that run on Linux. Cloud Foundry currently supports Ubuntu Trusty and CentOS 6.5 on vSphere, AWS, OpenStack, and vCloud infrastructures.

Degree of Statefulness - Runtime state includes caching and Sessions. Coordination of state across peer instances impacts performance. Any application that requires persistent heavy data processing. Apps that require large-scale stateful connections to external services. Use of frameworks that persist client state on the server side like JSF. Apps that keep a large amount of critical runtime state within the JVM i.e. stateful apps are bad candidates to the PaaS. Ideally all persistent state should reside outside the app in a HA data persistence tier.

File System - Cloud app instances are ephemeral. Local file system storage is short-lived. Instances of the same app do NOT share a local file system.Avoid writing state to the local file system. Does the app rely on a persistent local filesystem or storage ?

Transactionality - 2PC Transactional commit does not work across multiple distributed REST services. Prefer BASE(Basically Available, Soft state, Eventually consistent) over ACID(Atomicity, Consistency, Isolation & Durability). Transaction managers typical write transaction logs to local file system and rely on persistent server identity - both of these are not available by design in CF. http://queue.acm.org/detail.cfm?id=1394128

Configuration - Configuration should live external to the deployed app. Config must come from the Environment via environment variables via external configuration server. Apps that have embedded property files or one war/ear file per environment require the refactoring of configuration code before running on the PaaS. We recommend one code base tracked in revision control across multiple deploys. Can you deploy to multiple environments with single codebase.

Web Frameworks : Does the app leverage Java EE or Spring Frameworks and if so are the libraries bundled within the app or come with the app server. If there is strong dependence on an application server like WebSphere to provide dependencies or if proprietary application server APIs are used then that forces the app to use a particular buildpack or worse refactoring to start embedding the libraries within the app.

Startup/Shutdown Characteristics: Prefer apps that start cleanly without a lot of upfront initialization, coordination and shutdown without the need for comprehensive cleanup of state.
Is a startup/shutdown sequence necessary for your app to run effectively. Avoid creating any process instances outside the staged runtime.

Scalability of the App - Does the app rely on X-axis(horizontal duplication, Scale by cloning), Y-axis (functional decomposition, scale by splitting different things) or Z-axis (data partitioning, scaling by splitting similar things) ?. Design app as one or more stateless processes enabling scale-out via process model.Which other dependent services needs to be scaled when the app is scaled ?
Dependencies - How are external dependencies wired into the app ? Are dependencies isolated and explicitly declared. http://microservices.io/articles/scalecube.html

Inbound Protocols - Only a single inbound port is open to an application. Only protocol supported by CF is HTTP/HTTPs/WebSocket. No other protocols like RMI or JMX will work (unless tunneled over HTTP) on the inbound side. HTTPS is terminated at the load balancer. App that relies on transport security at the server will need to be modified. Apps that hold persistent long running socket connections to the server will not work, with apps using WebSocket being the only exception. Any sort of direct peer-to-peer messaging or custom protocol will not work since the warden containers are configured for ingress only over HTTP.

Logging - Are there multiple log streams ? Is there a strong requirement on local persistence of the logs ? Treat Logs of the app as event streams. Prefer console based logging to file based logging. Configure app server logging to log to the console (stdout/stderr) and thereafter drain to a long-term log analytics solution. Logs written to the local file system in CF are not persisted.

Rollback - Immutable code with instant rollback capability. Can the app tolerate multiple versions ? Can the app be reverted to a previous version with a single action ? Does deployment of the app require orchestration of multiple steps.

Security - Does the app rely on network centric or app centric security model ? We recommend relying on on application Layer 7 security rather than network security based on Firewall rules. How are application users authenticated and authorized. Is a Federated Identity and Authorization solution in place ? In CF, outbound egress from the app is controlled by application security groups applied to the warden container. You will need to configure whitelist rules for the services bound and protocols used for outbound communication.

Application Size - cf push times out for apps bigger than 250MB. Staging of apps that exceed a certain size becomes a big drain on resources for network marshalling/unmarshalling as well as runtime memory footprint. Keeping the droplet smaller results in faster deployment. client_max_body_size can be used to changed the max upload size on the Cloud Controller.

Performance & Monitoring - Can the app exhaust CPU under load? Are there any concurrency bottlenecks ? Can the app be instrumented for metrics ? Does the app provide a health check/actuator support in production. Does the app leverage app correlation IDs for distributed requests & responses. Does the underlying platform provide basic monitoring(CPU, Disk, Memory) and  recovery (Spawning of exited instances) and instant(manual/automated) scaling. Can the logging and admin  operations be enabled via runtime switches aka curl requests ?

Developer Experience - Does app architecture and web frameworks facilitate local development and cloud production. Does the app ruin on runtimes that run equally locally and in the cloud. Does the app use (Spring) profiles to encapsulate conditional app. configuration.

Cloud Native Architecture - Does the app compensate for the fallacies of distributed computing by leveraging framework libraries like Netflix OSS and/or SpringCloud OSS ? Does the app leverage zero downtime practices of forward and backward schema compatibility for REST microservices and databases. Can one service be deployed independently of other services. Is lockstep deployment of physical war's needed to manifest a logical app ?

Data Tier Services/Databases - Any persistence employed by the app has to be synchronized across datacenters and availability zones. The data tier/ Stateful tier of the app has to be managed in lock-step with the stateless tier. Focus has to be put on live content and data migration along with CAP theorem issues. App code has to be forward and backward compatible with schema changes in the data tiers.

For more on the exact process i.e. the HOW of migrating apps to the Cloud Foundry check this Pivotal blog post from Josh Kruck http://blog.pivotal.io/pivotal-cloud-foundry/features/how-do-i-migrate-applications-to-pivotal-cloud-foundry

How do I size my Cloud Foundry Production Deployment - shekel to the Rescue

Blog Post under construction - Due Date July 17 

Automated Backup and Restore of Cloud Foundry - cfops in Action

https://github.com/pivotalservices/cfops

Use cfops v2.x.x+

- This is simply an automation that is based on the supported way to back up Pivotal Cloud Foundry.

Download the latest version here: https://github.com/pivotalservices/cfops/releases/latest