Saturday, November 3, 2012

SOA 11g MDS - Share Common Artifacts

As promised on my previous post, I am detailing the step-by-step instructions which allows for sharing of common artifacts (schema files/WSDLs) among SOA projects and more importantly helps in loose coupling of service bindings & remove service dependencies.

A typical use case is where the abstract WSDL definition of an external service can be stored in MDS and referenced from the SOA composite which will loosely couple the services and also guard against 'SOA Composite Invalidation' issues.

SOA 11g provides us with MDS repository (both file based & DB based) where common artifacts can be stored and referenced by all projects in the SOA infrastructure.

File based MDS is used during design/deploy time where as DB based MDS is used at run-time by the SOA infra. A configuration file named adf-config.xml that is part of every SOA project manages this partition mapping to be used by JDeveloper during design/deploy time.


All SOA projects that are deployed on the SOA infrastructure are stored in the MDS repository. If you create a connection from the JDeveloper 'Resource Palette' to the SOA MDS, there will be a 'deployed composites' folder which will contain all deployed artifacts.

Connection Information for the MDS connection:
username: XXX_MDS (usually DEV_MDS)
password: xxxxxxxxx
MDS Partition: soa-infra


The file based MDS repository is generally the 'seed' partition under a typical JDeveloper install;

<JDev_Home>\integration\seed


Under this partition there is a root folder (soa) available by default which contains all the configuration files, schemas & WSDLs required by various different SOA components during design time. These files are also pre-deployed on the database MDS repository.

We are now free to create custom folders under the 'seed' partition to hold artifacts that are common references in a SOA environment. Generally, we create an 'apps' folder under the 'seed' partition which in turn holds various folders to hold various artifact types;
 Once the artifacts that needs to be shared/referenced are placed under the appropriate namespace (in this case 'apps'), we can let the SOA project in JDeveloper to point to MDS for these artifacts from a common location.

adf-config.xml configuration:


As noted earlier, adf-config.xml is an important configuration file which holds the details regarding the MDS (both file based & DB based). By default, this file declares the 'seed' partition & the metadata store for 'soa' namespace.

This file can be found under <YourApplication>\.adf\META-INF.
Alternatively, in JDeveloper, expand the 'Application Resources' panel; drill through Descriptors -> ADF META-INF to find this file


Step 1: Just add the metadata store namespace for '/apps' in additions to default '/soa/shared' namespace - This is to let the JDeveloper reference the artifacts under the '/apps' during the design time from the file system metadata store.

<adf-mds-config xmlns="http://xmlns.oracle.com/adf/mds/config">
    <mds-config xmlns="http://xmlns.oracle.com/mds/config">
      <persistence-config>
        <metadata-namespaces>
          <namespace path="/soa/shared" metadata-store-usage="mstore-usage_1"/>
          <namespace path="/apps" metadata-store-usage="mstore-usage_1"/>
        </metadata-namespaces>
        <metadata-store-usages>
          <metadata-store-usage id="mstore-usage_1">
            <metadata-store class-name="oracle.mds.persistence.stores.file.FileMetadataStore">
              <property name="metadata-path"
                        value="${oracle.home}/integration"/>
              <property name="partition-name" value="seed"/>
            </metadata-store>
          </metadata-store-usage>
        </metadata-store-usages>
      </persistence-config>
    </mds-config>
  </adf-mds-config> 


Step 2: In the SOA composite, reference the common artifacts from MDS as shown below;

 <import namespace="http://xmlns.oracle.com/bpmn/bpmnProcess/CreditCardProcess"
          location="oramds:/apps/orderprocess/common/CreditServiceProcess.wsdl"
          importType="wsdl"/>



Now that we have successfully referenced the SOA project to refer the file based MDS it is barely enough only during design time. For deploy time & run time, these artifacts must be made available in the DB based MDS.

There are two methods available for us to deploy the artifacts in SOA DB based MDS;

Approach 1: Ant based script which allows to deploy the artifacts into SOA MDS. I am not going to get to the details of this as this has been blogged in detail by Edwin Biemond; Reference below;

http://biemond.blogspot.com/2009/11/soa-suite-11g-mds-deploy-and-removal.html

Approach 2: For a simple JDeveloper based SOA deployment follow the steps below;

Step 1: Create a new application (Generic Application)

Step 2: Name the application (eg., OrderProcessMDSApp) and click 'Next'
 Step 3: Provide a name to the project (eg., OrderProcessMDSProj) and click 'Finish'

 Step 4: Now, right click on the OrderProcessMDSProj and choose 'Project Properties'. In the 'Project Properties' dialog, choose 'Deployment' and create a new deployment profile by clicking on 'New...' button

 Step 5: In the 'Create Deployment Profile' dialog, leave the default 'Archive Type' to 'JAR File' and name the deployment profile (eg., soamdsarchive) and click 'OK'

 Step 6: This will open up the 'Edit JAR Deployment Profile Properties' dialog. Click on the 'Contributors' section under 'File Groups -> Project Output'.
Step 7: Click on the 'Add..' button and choose the folder where the 'content to be deployed' is present. Here the '/apps' namespace under the file based MDS can be chosen where the content is already placed & available.
Step 8: Now, click on the 'Filters' section and choose the artifacts under the '/apps' that you would like to deploy or upload to MDS database repository
Step 9: Click on OK to complete the deployment profile creation process. Now, we have created a project JAR deployment profile which contains the artifacts to be deployed on to the SOA MDS. Now, we will have to create a application deployment profile of 'soa bundle' type which will be deployed to the soa-infra.

Step 10: From the 'Application Menu', choose 'Deploy' -> 'New Deployment Profile..' option
Step 11: Under the 'New Gallery' go to 'Deployment Profiles' choose 'SOA Bundle' and click OK
Step 11: Under the 'Create Deployment Profile', provide a name to the application deployment profile (eg., soabundle1) and click OK.
Step 12: In the 'SOA Bundle Deployment Profile Properties' go to 'Dependencies' and choose the JAR archive that we created earlier eg., soamdsarchive and click OK
Finally, deploy the application to the soa server. This will ensure that the artifacts under the '/apps' metadata namespace is deployed on to the SOA MDS database repository.
You can verify the same from the SOA MDS connection created under the 'Resource Palette'. This should now reflect the 'apps' folder at the same level as 'soa' metadata store.

Now, if you try to deploy the SOA project which references MDS you may hit a deployment error complaining that the resource referenced is not found. This is because the JDeveloper consults adf-config.xml configuration file during design & deploy time (refer back to the initial architecture diagram). Since there is no information about the DB based MDS in the adf-config the deployer complains.

To fix this, go back to the adf-config.xml file and add the DB metadata store information so that the deployer can read & locate the referenced resources from MDS.


<adf-mds-config xmlns="http://xmlns.oracle.com/adf/mds/config">
    <mds-config xmlns="http://xmlns.oracle.com/mds/config">
      <persistence-config>
        <metadata-namespaces>
          <namespace path="/soa/shared" metadata-store-usage="mstore-usage_2"/>
          <namespace path="/apps" metadata-store-usage="mstore-usage_2"/>
        </metadata-namespaces>
        <metadata-store-usages>
          <metadata-store-usage id="mstore-usage_2">
            <metadata-store class-name="oracle.mds.persistence.stores.db.DBMetadataStore">
              <property value="DEV_MDS" name="jdbc-userid"/>
              <property value="xxxxxxx" name="jdbc-password"/>
              <property value="jdbc:oracle:thin:@hostname:1521:orcl"
                        name="jdbc-url"/>
              <property value="soa-infra" name="partition-name"/>
            </metadata-store>
          </metadata-store-usage>
        </metadata-store-usages>
      </persistence-config>
    </mds-config>
  </adf-mds-config>

That completes our configuration and eventually this elaborate post. Now, the deployment will be smooth as we have referenced the DB MDS in the adf-config.xml file.

Hope this post helped you understand & implement the SOA MDS related activities. Let me know your feedback through comments.

34 comments:

  1. excellent ... thanks a lot :) . worked like a charm !!

    ReplyDelete
  2. Congratulation Sathya, i'm not using any Oracle products. But i have general question to ask on this topic.

    Is it possible to apply this on other products like Apache Ode BPEL engine. Where and how do you think we can specify concrete service properties.
    If you not familiar with Apache ODE, just ignore it and just consider a normal application that use abstract web services.

    Thanks

    ReplyDelete
    Replies
    1. Hi Themba,

      Thanks for your comment. I have read about Apache ODE but don't have a hands-on experience. In a normal Java/JEE application, generally to consume web services we use SOAP/WSDL engines such as Axis2 to generate the service classes to access/consume web services. And there is no need to use an abstract WSDL as the Java service (calling the web service) would still start even if the referenced webservice is unavailable - however, at runtime it would throw an error. Where as in a SOA engine which is primarily service oriented, the Oracle SOA engine would try to lookup & see if the service definitions are available during service startup phase itself to ensure that the service composite will work properly. Hence the explicit use of abstract service definitions. Hope it helps.

      Delete
  3. Hi Sathyam,

    Can you explain what is common schemas?? and how to use it?? Also what is the purpose of using this common schemas.?

    ReplyDelete
    Replies
    1. What I meant specifically by common schemas/artifacts is that something which may be used/referred more than once by one or more processes. The purpose is to reduce the usage/inclusion of these common schema(s) in all the referred places/projects.

      Delete
  4. Hi Sathya,

    Will you please let me know how to copy files from central MDS to local MDS connection.


    Thanks
    Kiran

    ReplyDelete
    Replies
    1. Kiran,

      I am not sure what you refer to as "central MDS" & "local MDS". As far as MDS is concerned, it maintains the instance state/metadata about SOA instances on the server in a database. This post exactly talks about how to deploy artifacts into MDS.

      Delete
  5. This Report Related to Above Topic" Services Oriented Architecture (SOA) Middleware Market(http://www.researchmoz.us/services-oriented-architecture-soa-middleware-market-shares-strategies-and-forecasts-worldwide-2013-to-2019-report.html) New market Research Report available at Researchmoz.us

    For More information Contact: sales@researchmoz.us .

    ReplyDelete
  6. Excellent post Sathya, nice work.

    ReplyDelete
  7. Sathya,

    That was a very good explanation of a complex and important feature of SOA. Thanks a lot for the post!

    Regards,
    Srinath

    ReplyDelete
  8. Hi Sathya,

    Thanks it is a Very Good Article.

    I have few questions here

    1)Can i completely copy the wsdl from browser and save in MDS? This wsdl will have hostname:port and no idea will it have any bad effect?
    2) I have four instances(dev,test,qatest,production) how can i move this Jar from one instance to other. Do i create the new jar for each instances as the wsdl in MDS is referring to hostname:port.
    3)Can i place xsds and wsdl in same folder in MDS? I have schemas that imports several other schemas? How does it work here?


    Thanks a lot.

    ReplyDelete
    Replies
    1. Hi,

      1. You should ideally store the abstract WSDL and not the concrete WSDL in MDS. Please take a look here - http://sathyam-soa.blogspot.in/2012/10/abstract-wsdl-vs-concrete-wsdl.html
      2. You should look at MDS migration - to move MDS artifacts between various instances
      3. Look at the WSDL & XSD imports - You should place them in the similar folder structure (relative paths).

      Delete
  9. Hi Sathya
    I have been following your blogs quite often to see if you already have a fix for the issue being encountered. Well, I have a question which is/isn't related to this post.
    I am using a build tool for the deployment and I want to modify the build.xml to replace the file specific for the environment.
    In short, I want to do is in the build.xml have some logic that says “if file exists ending in .dvm_${deployment.plan.environment} (.dvm_pc), replace the file of the same name in the working directory that ends with just .dvm”.
    Can you please help/guide me how to achieve the same. I have looked at several posts but couldn't find any wherein the artifact specific to the environment is being replaced during deployment.
    Any help or suggestion would be highly appreciated.
    Thanks
    Abhinav

    ReplyDelete
  10. Hi Satya, Nice article.
    Can you please tell me how to deploy soa bundled application other than using JDev or by using console. How can I undeploy soa bundle from repository. Thanks in advance.

    ReplyDelete
  11. Hi Satya

    Nice article.Can you please explain me what is MDS Explorer and how can we Deploy artifacts by using MDS explorer

    ReplyDelete
  12. Hi sathya,

    Excellent Post. Helped very much.

    ReplyDelete
  13. Hi Satya

    Is it possible to deploy files to soa/shared namespace in MDS?

    ReplyDelete
    Replies
    1. MDS repository is leveraged for file/artifact reuse for reasons mentioned in this post. Yes, you can deploy files to your SOA MDS.

      Delete
  14. Hi Sathya ,

    Can you please share some docs or useful links on updating SOA code from soa 11.1.1.6 to SOA 11.1.1.9 .

    Thanks in advance

    ReplyDelete
    Replies
    1. Please refer to the upgrade documentation: https://docs.oracle.com/middleware/11119/core/ASINS/start.htm#CIHGJFGJ

      Delete
  15. Hi I have installed SOA on a guest machine on VMware, and Jdeveloper on HOST machine. When I tried to use SOA resources, a WSDL, then there an error which says that WSDL not found, as hostname (demo.oracle.com) of guest

    ReplyDelete
    Replies
    1. Seems like an access issue. How are you running your VM - NAT / bridged mode? Check to see if you can ping your VM from your local. If on NAT mode, ensure you do the port mappings between your local and VM.

      Delete
  16. Replies
    1. Thanks Ayush for your comments. Glad it helped.

      Delete
  17. Hi Sathyam,

    We have worked on couple of SOA projects where we never chose to go with DB based MDS.
    i.e. We used file based MDS only available by default while design, deploy, run time of
    our SOA implementation which went fine. My question is that can we go with File based
    MDS safely or when to use DB based MDS to store project artifacts and why?

    Thank you for such detailed post and clear understanding about MDS.

    ReplyDelete
    Replies
    1. There are a few reasons why you will appreciate the use of a DB based MDS. For one, making dynamic changes is possible only with DB based MDS. For eg., you have a business rule component with a bunch of rules. If you have to modify a certain business rule dynamically at runtime in production, you can do that. You won't be able to achieve the same if you are using a file store. DB is much robust and secure as opposed to a file store. So instance recovery etc. is much faster and smooth in DB based MDS. I recommend all of my customers to use a DB store starting SIT, UAT and through Production. For dev, File is cheap and easy to work with. Hope that helps.

      Delete
  18. Nice post satya
    Is it possible to add Human Task,Rules to MDS. if yes how to upload and how to call them into BPEL

    ReplyDelete
    Replies
    1. MDS is the dehydration store. By default, SOA is long-running and leverages MDS (database) as dehydration store for various activity touchpoints. Human Task by nature is automatically dehydrated. Business Rules can work with both file based / DB based MDS, however, DB provides a bunch of benefits like dynamic modifications to rules @ runtime wihtout redeployments. Most of the components in Oracle SOA (included human tasks, business rules) can be designed to live on their own as distinct services. For eg., you can choose to have business rules exposed as services that can triggered/invoked from multiple SOA processes. Same applies to human tasks. Typically if you have a business rule/human task that is common and consumed by various SOA processes, you must separate them in different SOA processes. Hope that helps.

      Delete
  19. Hi Sathya,

    Nice blog. I am new to SOA. The one thing that I noticed is that putting the database credentials in adf-config.xml doesn't it make environment specific? The situation which we were trying to avoid. How to bypass that?

    Regards,
    Dipankar Banerjee

    ReplyDelete
    Replies
    1. Hi Dipankar, Thanks for your comments. adf-config is a configuration file that ideally must be part of the deployment artifacts. In a typical scenario, the deployment build manager must pick the right soa bundle to deploy to various environments. Think of this as a property file which supplies values to your SOA deployment @ runtime. Hope that helps.

      Delete