SOA 12c Compact Domain DB Based MDS

With the release of SOA 12c, developers can now have full-fledged SOA 12c running on their desktops/laptops on the integrated WebLogic server. This is a great news for development community - as one can download the SOA/BPM 12c quickstart which comes pre-configured with JDeveloper and integrated WLS running SOA/BPM.

Some developer productivity benefits include;
1) You don't need a license for quickstart installs (for development & evaluation purposes)
2) Single jar that will install & configure WebLogic domain, SOA, JDev and everything you need to get going

However, there are some limitations with the default quickstart install. For one, the quickstart installation runs on top of Java Derby DB; and the SOA MDS is a file-based repository by default. Although for normal development purposes this may not be a big challenge, it could be an issue if you want to leverage some features such as;

1) Run-time modification of business rules
2) B2B, MFT & ESS
3) SOA Composer
4) BAM
5) BPM Composer etc.. which require a Oracle database to run.

However, there are other ways to configure your development environment to run on top of Oracle database - therefore you can have a DB based SOA MDS.

In this post, I will illustrate how to install and configure a SOA 12c compact domain for development purposes.

Pre-Requisites:

1) Download Java 7. According to the certification matrix jdk 1.7.0u55+ is supported - here
2) Download Oracle XE 11g Database - here
3) Download SOA 12.1.3 quickstart distribution - here
4) Optional components such as B2B, MFT, can be downloaded if required
Download MFT - here
Download B2B - here

Install java 7 on your environment. Note: By default on windows jdk will choose a location under "Program Files". In certain cases, I have seen the space in folder location cause unknown issues. Ensure java is installed on a location without space in the folder name eg., C:\Java

SOA 12c Quickstart Installation:

Unzip the fmw_12.1.3.0.0_soaqs_Disk1_1of1.zip

Open terminal (command prompt) as administrator
Ensure JAVA_HOME and PATH environment variables are set and points to the jdk installed
set JAVA_HOME=C:\Java\jdk1.7.0_71
set PATH=%PATH%;C:\Java\jdk1.7.0_71\bin

Step 1:
Execute the following command to install the SOA 12c quickstart
java -D64 -jar fmw_12.1.3.0.0_soa_quickstart.jar

Step 2: Provide a oracle home where SOA 12c must be installed

Step 3: Ensure the validations are okay

Step 4: Review the installation summary - provides the list of features/components installed as part of the quickstart distribution

Step 5: Review the installation progress

Step 6: Once the installation succeeds, click Finish.

At this stage, quickstart can be used if you are not looking for features such as mentioned above. Just open the JDeveloper, right click on the IntegratedWeblogicServer to create and start domain. JDeveloper takes care of creating a WLS domain and configuring it automatically.

Follow through for a compact domain installation.

If you need additional components such as B2B, BPM or MFT now is the time to install them. Let's install B2B.

Installing Oracle B2B:

Step 1:
Unzip fmw_12.1.3.0.0_b2b_Disk1_1of1.zip and execute the following command to install B2B
java -D64 -jar fmw_12.1.3.0.0_b2bhealthcare.jar

Step 2: Choose the same Oracle home that was used to install the SOA 12c quickstart

Step 3: Choose B2B or Healthcare profiles depending on requirement. Let's choose B2B

Step 4: Follow the instructions and click Finish to close the B2B installation wizard

 

I haven't detailed the MFT installation. However, should you need, it is a very straightforward installation.

Installing XE 11g Database:

Unzip the OracleXE112_Win64.zip and follow the instructions to install XE database.

Step 1: Execute setup.exe

 Step 2: Accept the license agreement

Step 3: Choose the destination folder where XE must be installed

Step 4: Enter the SYS password

Step 5: Review and click Install

Step 6: Click Finish to finish the database installation

Creating RCU schemas for compact domain:

Step 1: Execute rcu.bat from the following location - $ORACLE_HOME\oracle_common\bin

Step 2: Leave the default selection on Create Repository and click Next

Step 3: Provide the database connect information and click next

Step 4: Ignore the warning. XE is not a "certified" database but will work fine in development environments.

Step 5: Default prefix is DEV. You can change this if you need. Select the SOA Infrastructure schema from the list which will select the required schemas

Step 6: Provide a password that will be used for all the RCU schemas

Step 7: Here you can select the size for the database profile. Default is small and should be fine for development environments. Optionally you can choose to enable/disable healthcare integration

Step 8: Follow the instructions and proceed with creation of database schemas for our compact FMW domain. Click close in the final summary screen.

Create and Configure Compact SOA Domain:

In this section let's create and configure a compact SOA domain.

To enable compact domain option, set the following environment variable before executing the config.cmd script

> set CONFIG_JVM_ARGS=-Dcom.oracle.cie.config.showProfile=true

> cd $ORACLE_HOME\oracle_common\common\bin

> config.cmd

Step 1: Choose the "Create a new compact domain" option and provide a domain name/location

Step 2: This is an important step where you must choose the templates with which the compact domain will be created. Choose all the required components. Note that the templates would show up based on the installations in the current domain.

Step 3: Choose a folder location where applications would reside

Step 4: Provide a administrator username and password.

Step 5: Choose Development mode / Production Mode (basically if you choose production mode, you will be prompted for the administrator username and password every time you start the weblogic server). Choose the JDK that we installed earlier which will be picked up by default.

Step 6:Provide the database details and click Get RCU Configuration. Once you get a success message proceed to next step.

Step 7: In the JDBC component schema page, select all the schema components and click next

Step 8: A test would run against the database for the schema connectivity. Once you get a success for all the components proceed further.

Step 9: For a compact domain with a "Admin" only server which will host all the middleware services leave the defaults on step 9 and click next. If you have a need to update the configuration such as adding managed servers, update default ports you can check the appropriate configuration and update.

Step 10: Click Create to create a compact SOA domain

Step 11: Ensure the domain creation succeeds and click Finish to close the config wizard.

That completes the creation of a SOA compact domain.

Now, if you start the weblogic server by issuing startWebLogic.cmd command, you will notice that the Java DB instance would start automatically. Since this is a compact domain and it will run off the Oracle XE database instance, we don't need the Java DB.

Start the weblogic with the noderby flag to prevent the Java DB from starting up.

> cd $ORACLE_HOME\user_projects\domains\compact_domain\bin

> startWebLogic.cmd noderby

Once the server starts up, establish connectivity to all the services to test the installation;

WLS Console: http://localhost:7001/console

EM Console: http://localhost:7001/em

SOA Composer: http://localhost:7001/soa/composer

B2B Console (if installed): http://localhost:7001/b2bconsole

BAM Console: http://localhost:7001/bam/console

One advantage with compact domain is that you can also create a integrated weblogic domain from JDeveloper which could be handy for a lot of development activities.

Are we ready for the cloud leap?

Cloud computing has come a long way from just being a buzz word. With the burgeoning of cloud companies and service providers across the planet, are we ready for the cloud leap yet?

Let’s reflect on a life-changing innovation that happened in the last decade – the evolution of smart phones. One can monitor health, pay utility bills, transfer funds, call a taxi and post an fb update on-the-go and in seconds. From just being a consumer focused platform that brought everything to our palms, it has matured into a first-class citizen of most enterprises today. It is hard to imagine appliction architectures today without a mobile strategy. Almost all customers I work with has a mobile strategy – at least in principle – and are getting their key end-user focused functions on mobile to keep pace with competition, if not outpace them. For instance, it is impossible for a bank to thrive in today’s aggressive environment without a mobile app that supports ‘Check Deposits’. Trendsetters like Uber, Instagram, Myntra, Hotel Tonight went mobile only strategizing their business model to consumer demand. That was the revolution of the last decade.

Is cloud the next big thing? ......

Read more @ Are we ready for the cloud leap?

Integrate OSB with Oracle MFT

Recently, I had an opportunity to work with a customer on MFT integration with OSB.

Some pain points were;
1) Ever-increasing FTP servers across their enterprise which was increasingly becoming tough to manage and maintain.
2) Custom code for compression/decompression logic before/after file transfers
3) Bunch of third-party utilities for scheduling file transfers
4) Java code exposed as API for PGP encryption/decryption

Obviously they had a lot of integration touch-points while putting together a file transfer solution - which also meant multiple points of failure and not to mention the maintenance/debugging nightmare if something were to go wrong.

One of the cool features of Oracle MFT is that it supports out-of-the-box integration with many technologies including OSB. For many OSB users, the following could be some of the use-cases where they could take advantage of MFT and reap benefits;

Traditional OSB implementations used File/FTP adapters for file transfers - No real visibility into the transfers and not the best of designs to manage/maintain.

With Oracle MFT exposed as a webservice it is now just a seamless webservice invocation - with the added benefits such as real-time visibility, OOTB scheduling, Compression, PGP encryption, ease of maintenance to name a few..

Use-Case 1: OSB triggers a MFT transfer

In this use-case OSB would trigger a MFT transfer. MFT would then fan-out the transfer;
a) Decompress, Decrypt and store in file system
b) Initiate a SOA process to do further processing (human task etc..)

Creating a MFT Source:

Step 1: Login to Oracle MFT console (http://hostname:port/mftconsole)

Step 2: Go to the Design tab. Click on the Sources and create a new OSB source as shown below;

Creating a MFT Target:

Step 3: Now, click on the Targets and create a new MFT target (This can be a webservice, SOA service, FTP, sFTP, File etc..).
In this case, we would be creating 2 targets - one SOA target and a File target as shown below;

Note: For the SOA target - I just created a simple SOA composite with MFT adapter...

Now. let's create a Transfer - nothing more than linking the source & target on a canvas.

Step 4: Click on the Transfers and create a new MFT. In the MFT canvas, drag and drop the OSBSrc to the source panel. Similarly, drag and drop the OrderSOATarget & PutFile targets to the target panel. Note that "decompress" and "decrypt" pre-processing actions are added to the targets. These are out-of-the-box capabilities that you can leverage with Oracle MFT.

That's all. Now just define a business service in OSB as usual that will trigger the transfer in Oracle MFT. You can obtain the MFT (OSBSrc) source webservice URL by just opening the source.

Use-Case 2: Oracle MFT invokes OSB proxy service

Here we will pick a file from the embedded FTP server (FTP & sFTP servers are embedded and comes out-of-the-box with Oracle MFT), and invoke a OSB target.

Step 1: Click on Sources and create a new source of type "FTP Embedded" as shown below;

Step 2: Click on Targets and create a new target of type "OSB" as shown below

Step 3: Click on the Transfers and create a new MFT. In the MFT canvas, drag and drop the FileSrc to the source panel. Similarly, drag and drop the OSBTgt to the target panel

Done. Now, let's see how to create a OSB proxy service that will consume the MFT transfer.

Step 4: Login to OSB console. Create a proxy service based on the WSDL available @ $FMW_Home/mft/integration/wsdl/MFTSOAService.wsdl (While source/target type is chosen as OSB or SOA, MFT exposes the web service based on this WSDL contract)

Note: Ensure that the proxy service URL and the MFT OSBTgt URL matches.

As easy as that.

Oracle MFT - The Managed File Transfer

With the 12c release for Fusion Middleware, Oracle released the long-awaited, feature-rich managed file transfer solution titled Oracle MFT. At the outset, some key value propositions of this product are;

1. Extremely Simple to use, build & configure
2. End-to-End visibility on file transfers
3. As with any other Oracle FMW stack of products, seamless integration options
4. High Availability MFT (leveraging WebLogic capabilities)
5. Highly Secure (Leverage WLS OPSS instead of typical FTP OS layer security)
6. OOTB embedded FTP & sFTP servers (Those of you looking to consolidate on ever increasing FTP & sFTP servers in your enterprise...)
7. Automate File Transfers using the built-in Enterprise Scheduler Service (ESS)
8. Transfer very large files (pass-by-reference) - limited only by disk space and network speed
9. Fan-out, chaining and custom callouts

Having worked with quite a few customers on this new Oracle technology, I can certainly tell that these features have generated a huge interest. Prior to Oracle MFT, it was really hard to find a product that could achieve all of the above requirements...

Let's delve little deep into these points...

Extreme Simplicity

No IDE to design/configure transfers.
Business user friendly unified web-based, highly intuitive graphical interface for design, monitoring & administration of all your file transfers.
Zero learning curve.

To create a transfer -> Just create your source and target - drag & drop them in the transfer canvas. You are good to go !! Incredibly, it's that simple...

Free-up developers: really? Yes.
How many times have you configured the cumbersome back-end scheduler jobs?
How many technical resources you hired/trained on various scripting languages just to manage and handle file transfers?

End-to-End Visibility

MFT console offers out-of-the-box monitoring dashboards with drill-down capabilities. Users can monitor overall file transfer metrics such as;

• Most Active Transfers
• Average, Min & Max Payload size, Transfer Speeds & Transfer Time
• Recent Errors/Failure Ratio
• Advanced File Search - based on file name, date, user, status etc..

You can also monitor a single file transfer instance graphically with fine grained details on the transfer. Additionally, Oracle MFT provides "Resubmit" option where users can resubmit a file transfer from various stages (Source, Transfer or Target).

Extensibility

Oracle MFT provides OOTB integration with the following source/target types in the current release. Obviously many more are coming in...

• FTP Embedded (Read/Write from/to the embedded FTP server)
• sFTP Embedded (Read/Write from/to the embedded sFTP server)
• FTP Remote (Read/Write from/to a remote FTP server)
• sFTP Remote (Read/Write from/to a remote sFTP server)
• File (Read/Write from/to the shared file storage)
• SOAP (Send/Receive to/from a SOAP based web service)
• SOA (Send/Receive to/from a SOA composite) - SOA 12c provides an MFT adapter. Optionally, you can also connect to a SOA/OSB 11g instance through webservice
• Service Bus (Send/Receive to/from an OSB proxy/business service)
• B2B (Connect to a B2B with MFT as a channel)
• Healthcare (Connect to Healthcare HL7)
• ODI (Connect to Oracle Data Integrator)

High Availability

Oracle MFT runs on Oracle WebLogic 12c and can be clustered. This provides organizations with an HA MFT capability. Oracle MFT also ships with an MFT proxy server that resides outside the firewall to redirect external requests.

Highly Secure

Since MFT runs on WebLogic, it leverages Oracle WebLogic OPSS.
What this means is that: all your FTP/sFTP users in your LDAP can now connect to MFT (obviously the access rights can be managed through the MFT console).

In stark contrasts to traditional FTP/sFTP servers, you don't have to create OS level users & manage access privileges. Optionally, for each transfer, you can configure Oracle MFT to allow access to payload only to specific users/groups.

In additions to these capabilities, users can leverage out-of-the-box compression, decompression, PGP encyption & decryption capabilities of Oracle MFT.

Consolidation

Many customers have very less control on FTP & sFTP server proliferation across their enterprise. More often than not, IT doesn't have full control / visibility on administration & management of the FTP servers. Oracle MFT with its embedded FTP & sFTP server clearly helps with consolidation of FTP/sFTP servers.

This not only reduces TCO tangibly, but also brings down significant costs associated with administration/management efforts.

Automate & Schedule Transfers

How many scheduling software tools do you have in your enterprise?
How difficult is it to integrate your scheduler with file transfers?
How much time & effort does your IT/development spend in configuring/scripting of scheduler services?
How much visibility does it offer?

And,

What if; you have one comprehensive & highly customizable enterprise scheduler service which is tightly integrated with all your file transfers, is extremely easy to configure/schedule through GUI and provides end-to-end visibility through a console?

With Oracle Enterprise Scheduler Service (ESS) now being a first-class citizen & tightly integrated, Oracle MFT just does that.

Large File Transfers

Oracle MFT supports file pass-by-reference. This means that your SOAP services do not have to pass file to your webservice inline if it beyond a certain size - configurable while designing the transfer. Oracle MFT also provides out-of-the-box compression & decompression capabilities. Using custom actions you can also batch/de-batch large files for fast processing.

Fan-out, Chaining & custom callout

Oracle MFT supports fan-out: which means it can send files to multiple targets simultaneously. Also another cool feature if MFT process chaining. You can chain multiple MFT transfers where one MFT transfer can be another MFT's source.

Additionally, Oracle MFT also supports custom callouts. This allows POJOs to be executed at source and target (pre/post process).

Ready for a happy file transfer with Oracle MFT? Let me know your feedback in the comments section :)