Monday, February 16, 2015

Integrate OSB with Oracle MFT

Recently, I had an opportunity to work with a customer on MFT integration with OSB.

Some pain points were;
1) Ever-increasing FTP servers across their enterprise which was increasingly becoming tough to manage and maintain.
2) Custom code for compression/decompression logic before/after file transfers
3) Bunch of third-party utilities for scheduling file transfers
4) Java code exposed as API for PGP encryption/decryption

Obviously they had a lot of integration touch-points while putting together a file transfer solution - which also meant multiple points of failure and not to mention the maintenance/debugging nightmare if something were to go wrong.

One of the cool features of Oracle MFT is that it supports out-of-the-box integration with many technologies including OSB. For many OSB users, the following could be some of the use-cases where they could take advantage of MFT and reap benefits;

Traditional OSB implementations used File/FTP adapters for file transfers - No real visibility into the transfers and not the best of designs to manage/maintain.

With Oracle MFT exposed as a webservice it is now just a seamless webservice invocation - with the added benefits such as real-time visibility, OOTB scheduling, Compression, PGP encryption, ease of maintenance to name a few..

Use-Case 1: OSB triggers a MFT transfer

In this use-case OSB would trigger a MFT transfer. MFT would then fan-out the transfer;
a) Decompress, Decrypt and store in file system
b) Initiate a SOA process to do further processing (human task etc..)



Creating a MFT Source:

Step 1: Login to Oracle MFT console (http://hostname:port/mftconsole)

Step 2: Go to the Design tab. Click on the Sources and create a new OSB source as shown below;


Creating a MFT Target:

Step 3: Now, click on the Targets and create a new MFT target (This can be a webservice, SOA service, FTP, sFTP, File etc..).
In this case, we would be creating 2 targets - one SOA target and a File target as shown below;

Note: For the SOA target - I just created a simple SOA composite with MFT adapter...



Now. let's create a Transfer - nothing more than linking the source & target on a canvas.

Step 4: Click on the Transfers and create a new MFT. In the MFT canvas, drag and drop the OSBSrc to the source panel. Similarly, drag and drop the OrderSOATarget & PutFile targets to the target panel. Note that "decompress" and "decrypt" pre-processing actions are added to the targets. These are out-of-the-box capabilities that you can leverage with Oracle MFT.



That's all. Now just define a business service in OSB as usual that will trigger the transfer in Oracle MFT. You can obtain the MFT (OSBSrc) source webservice URL by just opening the source.




Use-Case 2: Oracle MFT invokes OSB proxy service

Here we will pick a file from the embedded FTP server (FTP & sFTP servers are embedded and comes out-of-the-box with Oracle MFT), and invoke a OSB target.

Step 1: Click on Sources and create a new source of type "FTP Embedded" as shown below;

Step 2: Click on Targets and create a new target of type "OSB" as shown below

Step 3: Click on the Transfers and create a new MFT. In the MFT canvas, drag and drop the FileSrc to the source panel. Similarly, drag and drop the OSBTgt to the target panel

Done. Now, let's see how to create a OSB proxy service that will consume the MFT transfer.

Step 4: Login to OSB console. Create a proxy service based on the WSDL available @ $FMW_Home/mft/integration/wsdl/MFTSOAService.wsdl (While source/target type is chosen as OSB or SOA, MFT exposes the web service based on this WSDL contract)

Note: Ensure that the proxy service URL and the MFT OSBTgt URL matches.

As easy as that.

11 comments:

  1. Nice Post. i have one requirement to read 5GB file from dynamic location with dynamic file name and write in a location with dynamic folder name . Service should be exposed a web service through SOA . so that other entity can use this service to pass the location and file details. can you please help me one this, how to achieve it?

    ReplyDelete
    Replies
    1. This is certainly doable. You can chain MFT with OSB or SOA (BPEL). In your case you must invoke MFT from SOA/OSB - MFT Source Type must be SOA or Servicebus. MFT also exposes itself with <MFTHeader> where dynamic parameters can be passed.

      Delete
  2. Thanks for the post. Could you please help on these :-
    1) Have you had success with SOAP based source (Remote SFTP). I get a FILE_NOT_FOUND exception with this case.
    2) What sort of inputs we need to pass in the SOAP enveloper to the business service? I was planning to expose it via a proxy service and make it generic for use in multiple integrations.

    ReplyDelete
    Replies
    1. Arinjay,
      1) I am not sure if you are using SOAP source or Remote SFTP source. Remote SFTP is the source type for MFT to pick files from a remote SFTP server instead of the embedded SFTP.
      2) Need more details here. You must create MFT with servicebus source type and configure OSB to front MFT as detailed in this blog.

      Delete
  3. Are you able to decompress in the pre-processing activity using a File target. I get a decompression only allowed for SOA/OSB or ODI target. Decompression is only working in the post-processing action

    ReplyDelete
    Replies
    1. What release of MFT are you using? File decompression is available as a pre-processing activity.

      Delete
  4. I am getting following error while invoking from MFT to SOA

    MFTException [threadName=JCA-work-instance:JMSAdapter-4, errorID=cfe04205-e216-4c2c-b42a-f0dad0ee0b0c, errorDesc=MFT-5401_MFT Service could not deliver message to Web Service endpoint http://localhost.ILMS.com:port/soa-infra/services/default/MFTSOAPrototype/mftService., cause=Client received SOAP Fault from server : Unable to find operation: unknown]

    ReplyDelete
  5. In your first Use-Case 1: OSB triggers a MFT transfer, don't we need to create a Proxy Service on the OSB?

    ReplyDelete
    Replies
    1. You mean - to invoke the OSB service? That depends on how you design your OSB service. Use case 1 described is for MFT invocation from OSB. How the OSB service is triggered is black box for this usecase...

      Delete
  6. In osb can we use more than one http based proxy service which calls one business service??
    In my case i have 3 http based proxy which is calling one BS?
    is this possible coz in my scenario BPEL is triggring 2nd OSB service but while invoking those its getting failed out in invoke activity with null pointer exception...for 1st proxy it is working fine
    could you please kindly help me with this??

    Thanks

    ReplyDelete
    Replies
    1. Hi Falguni,

      Yes, you can have multiple proxies pointing to the same Business Service. That's a major advantage of OSB. You can have each proxy with a different protocol (SOAP, REST etc..) used by different sources. There is very little information on the issue for me to suggest a solution for your "null pointer exception".

      Delete