tag:blogger.com,1999:blog-55493520109198628572024-03-20T20:39:36.219+05:30Rohan Lopes BlogRohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comBlogger47125tag:blogger.com,1999:blog-5549352010919862857.post-71699428576755101722023-07-08T23:22:00.000+05:302023-07-08T23:22:01.299+05:30Journey towards launching: Follow My Church Mobile App - (iOS & Android) <div style="text-align: left;">
At the start of 2023 I wanted to work on a pet project to implement Cloud
Native App. <br />Following a series of brainstorming sessions, I reached
the decision to develop an application specifically designed for
churches.
</div>
<p>
<b>The primary objective is to create a multi-tenant app that enables each
individual church to independently manage its own content, while allowing
users to access and view the relevant content for their respective churches
through a mobile app.</b>
</p>
<div style="text-align: left;">
I will attempt to guide you through the entire process of implementing this
pet project.
</div>
<div style="text-align: left;">
<br />***As this project was undertaken as a personal endeavor, the objective
was to minimize costs by predominantly utilizing open source technologies and
free tier products***
</div>
<div style="text-align: left;"><br /></div>
<div style="text-align: left;"><br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlysl0CfnlFiuT-VlgQMkjrkZYn17gWn0Uh_2p--0VD2qA-lcXbHW5XV2CWX6ZO2jal_y-honm6Q8UV69-vsPk843erpHYrcfARy74WAmBirqBmNxR7jhVkijjQvSgIS1a4AaZueGwqYGKdhCCh9x53wVratBFB_U8m8IztQIic6_Udh8NHWjb8Pb4dz5E/s14404/Follow%20My%20Church.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="7618" data-original-width="14404" height="338" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlysl0CfnlFiuT-VlgQMkjrkZYn17gWn0Uh_2p--0VD2qA-lcXbHW5XV2CWX6ZO2jal_y-honm6Q8UV69-vsPk843erpHYrcfARy74WAmBirqBmNxR7jhVkijjQvSgIS1a4AaZueGwqYGKdhCCh9x53wVratBFB_U8m8IztQIic6_Udh8NHWjb8Pb4dz5E/w640-h338/Follow%20My%20Church.jpg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><b>Follow My Church - High Level Components </b></td></tr></tbody></table><br /><div style="text-align: left;"><br /></div>
<h3 style="text-align: left;">
Phase 1: <b>Develop</b> <b>Prototype with App Features for feasibility study</b>
</h3>
<p>
Created a prototype which will help visualize the To-Be product and its
features.
</p>
<p>
<span id="docs-internal-guid-62b6c139-7fff-ad12-37d1-92b3f2ee86fd"><img height="200" src="https://lh5.googleusercontent.com/E7ShgdAynlBqLVrcLTnDrD9NvoUA4bHUCQONpo_pd14o_r9z6-3aFjKGHq3ko1Bl1s4VJ1om023nURMlJj3f0oD8nMAheU779pYtWjoKuvsMPywKP9ZhnqytO62aRNYMd93FNV9jNnn94CRk4kKSFXGtbg=w93-h200" width="93" /> </span><img height="200" src="https://lh4.googleusercontent.com/D4d_oi_ONDjXmXDNprmo_WRbuE98kANFsAmABjDmav2plJ-fa6B-fFVNJwYL9Oj7BcM79pmVyQS4DMm2ePN2f9i21Sl3BLWoAc4-FEyA0JpHly6mDfOMu_n1qj716naZWrtOfn9t5q24Mzljqo_Crk7BwA=w93-h200" width="93" /> .<img height="200" src="https://lh4.googleusercontent.com/3WiXjEzBDnYr0hN_DCXjyhJJd9_5xt8U_n92liWoKEgrRdM0BwF8OVNMnDDuK5x1r17gKEXQqCUtgpOcI7GPb3rjM8fEpm44JPTJ-A__HxrwUdVnPgjlTGJIi0rojM_d-5p3SM9qv7aB4URXiagF0q69fA=w93-h200" width="93" />....
</p>
<div style="text-align: left;">
<b>Tools & Languages:</b>
<b><a href="https://www.figma.com/" target="_blank">Figma</a></b>
</div>
<p><br /></p>
<h3 style="text-align: left;">
Phase 2:
<b>Find approach to implement a multi-tenant backend system to allow Church
Admin/Editor to manage its own contents.</b>
</h3>
<div style="text-align: left;">
In this scenario, I had to devise an approach for developing a backend web
application that functions as a <b>Software-as-a-Service (SaaS) product</b>.
This application enables individual churches to onboard and independently
manage their own content, such as
<b>articles, announcements, events, and donation links.</b>
</div>
<div style="text-align: left;">
<b><br /></b>After extensive research and trials, I selected
<a href="https://payloadcms.com/" target="_blank">PayloadCMS </a>an open
source headless Content Management System.
</div>
<div style="text-align: left;">
<br />I very much impressed with PayloadCMS, it allowed me to implement the
multi-tenant backend app with in a month time. It also comes with out of the
box REST & GraphQL API's.
</div>
<div style="text-align: left;">
<br />PayloadCMS support NoSQL MongoDB. Thus, selected MongoDB Atlas with
Serverless version which was much cost effective and easy to setup.
</div>
<div style="text-align: left;">
<br /><b>Tools & Languages:</b>
<b>React JS, <a href="https://payloadcms.com/" target="_blank">PayloadCMS</a>,
<a href="https://www.mongodb.com/" target="_blank">MongoDB Atlas<br /></a>Production Site: </b><a href="https://church.followmychurch.org/admin" target="_blank">Follow My Church Backend</a>
</div>
<p><br /></p>
<h3 style="text-align: left;">
Phase 3: <b>Theming - Logo's, Fonts, Color's Etc.</b>
</h3>
<div style="text-align: left;">
To achieve a more polished and professional appearance for my app, I initially
chose my own themes, but the results were not as impressive as I had
hoped.
</div>
<div style="text-align: left;">
<br />Consequently, I made the decision to seek assistance from a freelancer
who aided me in designing logos, selecting fonts, colors, and other visual
elements for both the backend and mobile app.
</div>
<div style="text-align: left;">
<br /><b>Credit:</b> <a href="https://anjutuscano.blogspot.com/" target="_blank">@AnjuTuscano</a> - Please visit her blog for more information.<br /><a href="https://anjutuscano.blogspot.com/2023/05/ui-elements-for-follow-my-church-app.html" target="_blank">UI Elements for Follow My Church App</a>
</div>
<p><br /></p>
<h3 style="text-align: left;">
Phase 4: <b>Selecting the Cloud Platform - Google Cloud Platform</b>
</h3>
<div style="text-align: left;">
It is obvious I will need the cloud platform to host the backend/Frontend apps
and services. For years, I have been experimenting with Google Cloud Platform
and since I was pretty comfortable using GCP products, I decided to implement
it on GCP.
</div>
<p>Below are the <b>key </b>GCP services used in this project:</p>
<p></p>
<ul style="text-align: left;">
<li>
<b>Cloud Run</b> - A fully managed platform for containerized Apps.
</li>
</ul>
<ul style="text-align: left;">
<ul>
<li>Service Hosted: </li>
<ul>
<li><b>PayloadCMS</b> as a web and backend services</li>
<li>Tagged with Custom Domain Name</li>
</ul>
</ul>
</ul>
<ul style="text-align: left;">
<ul>
<li>Reasons for selection:</li>
<ul>
<li>
It allow to host your docker container in serverless mode, and pricing
is based on the number of transactions. Since, I am not expecting much
traffic at the start, I didn't wanted to pay for dedicated server and
using <b>Kubernetes </b>was overkill.
</li>
</ul>
</ul>
</ul>
<ul style="text-align: left;">
<li>
<b>Cloud Storage - </b>To<b> </b>store uploaded Documents & Images
</li>
<ul>
<li>
All the content i.e. uploaded by Church's are saved with applied security
on Cloud Storage.
</li>
</ul>
</ul>
<ul style="text-align: left;">
<li>
<b>Cloud Artifact - </b>The Docker build generated via GitHub is pushed and
maintained in GCP Artifact storage for versioning.
</li>
</ul>
<ul style="text-align: left;">
<li><b>Cloud Functions</b> - To host serverless functions. </li>
<ul>
<li>Service Hosted:</li>
<ul>
<li>
Church Middleware Service which is implemented in NodeJS to orchestrated
calls from Mobile App to Backend CMS services with API security etc.
</li>
</ul>
</ul>
</ul>
<ul style="text-align: left;">
<ul>
<li>Reasons for Selection:</li>
<ul>
<li>
The middleware was implemented using <b>Node JS Framework</b> and it was
very simple to host it as a API using Cloud Functions.
</li>
</ul>
</ul>
</ul>
<p></p>
<div style="text-align: left;">
<i>***Approaches taken are towards using as serverless services as much
possible, as it is easy to scale on demand and incur no cost if you are a
startup.</i>
</div>
<div style="text-align: left;">
<i><br /></i>
</div>
<div style="text-align: left;">
<b>Tools & Languages:</b> <b>Node JS, Cloud Run, Cloud Function, Docker, Github</b>
</div>
<p>
<b><br /></b>
</p>
<h3 style="text-align: left;">
Phase 4: <b>Selecting technology for implementing mobile app</b>
</h3>
<div style="text-align: left;">
I want to use the framework which will allow me build app for all platform
i.e. Android, IOS, Web.
</div>
<div style="text-align: left;">
<br />After exploring, <b>React Native, Ionic, Flutter, </b>I decided to
implement it in Flutter because of the wide community support, cool widgets,
animation etc.
</div>
<div style="text-align: left;">
<br /><b>Tools & Languages: <a href="https://flutter.dev/" target="_blank">Flutter</a></b>
</div>
<p>
<b><br /></b>
</p>
<h3 style="text-align: left;">
Phase 5: <b>Selecting Mobile App Backed Services</b>
</h3>
<div style="text-align: left;">
To implement any scalable mobile app, we need a mobile app backend service
platform which will help us implement and scale on need basis.
</div>
<div style="text-align: left;">
<br />After researching and trying, <b>AWS Amplify and Google Firebase</b>, I
decided to go with <b>Google Firebase</b> for its simplicity, examples, use
cases and my familiarity with GCP platform. I also wanted to keep infra
limited to single cloud provider.
</div>
<div style="text-align: left;">
<br /><b>Tools & Languages:</b>
<b><a href="https://firebase.google.com/" target="_blank">Firebase </a>-
Authentication, NoSQL DB, Functions, Analytics, App Notifications
etc.</b>
</div>
<p><br /></p>
<h3 style="text-align: left;">
Phase 6: <b>Implementing a Middleware Service between Mobile App and Headless CMS</b>
</h3>
<div style="text-align: left;">
The Scenario was that the Church Admin's/Editor's will use the backend System
to publish the Articles. This Articles will have Fonts, Lists, Images.
</div>
<div style="text-align: left;">
<br />In order for Mobile app to render the Articles & Events, we needed a
Middleware service. Also, it was bit tricky to add the JWT Validation on
PayloadCMS, thus implemented the JWT validation in Middleware API.
</div>
<p>
<b><br /></b>
</p>
<h3 style="text-align: left;">
Phase 7: <b>Implementing Actual Mobile App and its Features</b>
</h3>
<p>The mobile app implemented has below features:</p>
<p></p>
<ul style="text-align: left;">
<li>List and View Articles</li>
<li>List and View Events</li>
<li>List and View Announcements </li>
<li>List and View Mass Timings</li>
<li>Donation Options</li>
<li>Search Churches</li>
<li>About My Church Info</li>
<li>App Notifications using Firebase Messaging</li>
<li>App Analytics using Firebase Analytics</li>
<li>Sign in with Google using Firebase Authentication</li>
</ul>
<p></p>
<h3 style="text-align: left;">
Phase 8: <b>Implementing CI/CD for Flutter Mobile App using Code Magic</b>
</h3>
<div style="text-align: left;">
I wanted an approach to build and publish the App in draft release on Google
Play. CodeMagic is the cool tool to set it up with your Github Repo.
</div>
<div style="text-align: left;">
<br />You can configure it either start on Git Check in or initiate it
manually.
</div>
<div style="text-align: left;">
<br /><a href="https://codemagic.io/start/" target="_blank">https://codemagic.io/start/</a>
</div>
<p><br /></p>
<h3 style="text-align: left;">
Phase 9: <b>Creating Marketing materials for publishing the App - Canva</b>
</h3>
<p>
In Order to publish app on Play Store, I needed to create the marketing video.
Canva was a quick option to create demo videos.
</p>
<p><br /></p>
<h3 style="text-align: left;">
Phase 10: <b>Signing up for Google Play and releasing beta version and sharing it with
Friends and Family</b>
</h3>
<div>
<b><br /></b>
</div>
<h3 style="text-align: left;">
Phase 11: <b>Submitting the App for approval on Play Store and publishing it.</b>
</h3>
<h4 style="text-align: left;"><br /></h4>
<h3 style="text-align: left;">
Phase 12: <b>Implementing the web site for users to know about the app. </b>
</h3>
<div style="text-align: left;">
Hosted the simple ReactJS Site to provide details about the App. The site is
hosted on Firebase Hosting.
</div>
<div style="text-align: left;">
<br /><b>Tools & Languages:</b> <b>Firebase - Hosting</b>
</div>
<div style="text-align: left;">
<b><br /></b><b><a href="https://followmychurch.org/" target="_blank">https://followmychurch.org/</a></b>
</div>
<p><br /></p>
<h3 style="text-align: left;"><b>Code Repo & Development tools:</b></h3>
<p></p>
<ul style="text-align: left;">
<li>Github</li>
<li>Github Codespace with Visual Studio</li>
<li>Android Studio for Flutter</li>
</ul>
<p></p>
<p><br /></p>
<h2 style="text-align: left;">
You can view the App by downloading it from Play Store.
</h2>
<p>
<a href="https://play.google.com/store/apps/details?id=org.followmychurch.app" style="margin-left: 1em; margin-right: 1em; text-align: center;" target="_blank"><img alt="" data-original-height="512" data-original-width="512" height="45" src="https://blogger.googleusercontent.com/img/a/AVvXsEivlkR8dUYQcjFR_RwXTGOCuLP7d4pVPR2zLU3Q6EJQvZDoY4vklZg0RwQpceCjcYXGXBlo4JlLloJX1Ow5QKUvF8eFCWXHSdw03lSEwOdTM_y-KK5sjoO3ttEDTHzTWIbxMpSIyHzWbnlRDK5de0ONNVnARgdRB8uibSaG13bKh2tfBu9seuhUPDgBSR4v=w45-h45" width="45" /></a>
</p>
<p>
<a href="https://play.google.com/store/apps/details?id=org.followmychurch.app" target="_blank">Follow My Church Google Play App</a>
</p>
<p><br /></p>
<p>If you want to check demo videos on App and Backend check below site.</p>
<p>
<a href="https://followmychurch.org/" target="_blank">https://followmychurch.org/</a><br />
</p>
<p><br /></p>
<p>P.S: iOS Version will launch soon..</p>
<p><br /></p>
Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-89519825915515732992022-12-09T04:32:00.000+05:302022-12-09T04:32:02.714+05:30Combine or Merge XML documents in Single XML using Boomi & Groovy<p></p><p class="MsoNormal" style="line-height: normal; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;"><span style="color: black; font-family: "Times New Roman",serif; font-size: 13.5pt; mso-fareast-font-family: "Times New Roman";">In this small post will
try to show how you can merge multiple XML Documents in Boomi. <o:p></o:p></span></p>
<p class="MsoNormal" style="line-height: normal; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;"><span style="color: black; font-family: "Times New Roman",serif; font-size: 13.5pt; mso-fareast-font-family: "Times New Roman";">Consider following
scenario where you have to merge Document 1 & 2 in a new Document 3<o:p></o:p></span></p>
<p class="MsoNormal" style="line-height: normal; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;"><u><span style="color: black; font-family: "Times New Roman",serif; font-size: 13.5pt; mso-fareast-font-family: "Times New Roman";">Document 1:</span></u><span style="color: black; font-family: "Times New Roman",serif; font-size: 13.5pt; mso-fareast-font-family: "Times New Roman";"><o:p></o:p></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><?xml version="1.0" encoding="UTF-8"?><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><tns:OutputParameters
xmlns:tns="http://sample.com"><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span><tns:Book><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span><tns:Names><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span><tns:Name>XZY
World</tns:Name><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span></tns:Names><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span><tns:Authors><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;">
</span><tns:AuthorID>12354</tns:AuthorID><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span></tns:Authors><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span></tns:Book><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 5.0pt; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"></tns:OutputParameters></span><span style="font-size: 10pt;"><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New"; font-size: 10.0pt; mso-fareast-font-family: "Times New Roman";"><o:p> </o:p></span></p>
<p class="MsoNormal" style="line-height: normal; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;"><u><span style="color: black; font-family: "Times New Roman",serif; font-size: 13.5pt; mso-fareast-font-family: "Times New Roman";">Document 2:</span></u><span style="color: black; font-family: "Times New Roman",serif; font-size: 13.5pt; mso-fareast-font-family: "Times New Roman";"><o:p></o:p></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><?xml version="1.0" encoding="UTF-8"?><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><tns:OutputParameters
xmlns:tns="http://sample.com"><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span><tns:Book><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span><tns:Names><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span><tns:Name>ABC
World</tns:Name><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span></tns:Names><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span><tns:Authors><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;">
</span><tns:AuthorID>53171</tns:AuthorID><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span></tns:Authors><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span></tns:Book><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 5.0pt; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"></tns:OutputParameters><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><o:p><span style="font-size: x-small;"> </span></o:p></span></p>
<p class="MsoNormal" style="line-height: normal; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;"><u><span style="color: black; font-family: "Times New Roman",serif; font-size: 13.5pt; mso-fareast-font-family: "Times New Roman";">To a Merged Document:</span></u><span style="color: black; font-family: "Times New Roman",serif; font-size: 13.5pt; mso-fareast-font-family: "Times New Roman";"><o:p></o:p></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><?xml version="1.0" encoding="UTF-8"?><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><tns:OutputParameters
xmlns:tns="http://sample.com"><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span><tns:Book><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span><tns:Names><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span><tns:Name>XZY
World</tns:Name><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span><tns:Name>ABC
World</tns:Name><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span></tns:Names><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span><tns:Authors><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;">
</span><tns:AuthorID>12354</tns:AuthorID><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;">
</span><tns:AuthorID>53171</tns:AuthorID><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span></tns:Authors><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"><span style="mso-spacerun: yes;"> </span></tns:Book><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 5.0pt; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New";"><span style="font-size: x-small;"></tns:OutputParameters></span><span style="font-size: 10pt;"><o:p></o:p></span></span></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: 0in; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: black; font-family: "Courier New"; font-size: 10.0pt; mso-fareast-font-family: "Times New Roman";"><o:p> </o:p></span></p>
<p class="MsoNormal" style="line-height: normal; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;"><span style="color: black; font-family: "Times New Roman",serif; font-size: 13.5pt; mso-fareast-font-family: "Times New Roman";">Merging this in Boomi
using Map shape is tricky because the elements are repeating and was not able
to make it work even using cached documents in Map. <o:p></o:p></span></p>
<p class="MsoNormal" style="line-height: normal; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;"><span style="color: black; font-family: "Times New Roman",serif; font-size: 13.5pt; mso-fareast-font-family: "Times New Roman";">Thus, wrote below Groovy
Script to implement the logic. <o:p></o:p></span></p>
<p class="MsoNormal" style="line-height: normal; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;"><span style="color: black; font-family: "Times New Roman",serif; font-size: 13.5pt; mso-fareast-font-family: "Times New Roman";"><o:p> </o:p></span><span style="font-family: Symbol; font-size: 13.5pt; text-indent: -0.25in;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;"> </span></span><span style="font-family: "Times New Roman", serif; font-size: 13.5pt; text-indent: -0.25in;">Firstly to simulate the multiple documents you
can create a sub process with a return shape. Since return shape wait for all
documents to arrive below actually returning them.</span></p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhK0FaWVo5bjDvzc8imKGvOii6X1VLjF-lPi9kc-eWCm8tveLQHh_ROoihq2x1LZjSjO12-rlqOvOUVwSArOILKKixewZCrKbqtpuyAt0fU1lvdhT42VMLW6log0BArCR2FDR6Q8VtokOT_GuF2p79N-t2JSa6i99Y-wFyOkmA9JwbTEBRFrNef8JmBNg/s621/Capture1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="207" data-original-width="621" height="214" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhK0FaWVo5bjDvzc8imKGvOii6X1VLjF-lPi9kc-eWCm8tveLQHh_ROoihq2x1LZjSjO12-rlqOvOUVwSArOILKKixewZCrKbqtpuyAt0fU1lvdhT42VMLW6log0BArCR2FDR6Q8VtokOT_GuF2p79N-t2JSa6i99Y-wFyOkmA9JwbTEBRFrNef8JmBNg/w640-h214/Capture1.JPG" width="640" /></a></div><br /><br /><p class="MsoNormal" style="line-height: normal; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;"><span style="font-family: Symbol; font-size: 13.5pt; text-indent: -0.25in;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;"> </span></span><span style="font-family: "Times New Roman", serif; font-size: 13.5pt; text-indent: -0.25in;">Next you can call the sub process and pass multiple
documents to Data Process shape with Groovy Script</span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgIJHyacS_xt7L5r-pSqExOJyuUTBH3Ns_itZQh-vcj1OdFpTlZmMx3gNWKq3_zC0wM3Kqll_v3ac4X19CERS5r2iWFQD7t2ndmc9AxLedAoE00JJc7x3tQ_IU9J6F2JputN-ZEq5E0Sr8xMLkD8CDyK5aTmr05k1RyGAUG2XSlbBjxjkULb-kRc4B1wQ/s675/Capture.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="113" data-original-width="675" height="108" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgIJHyacS_xt7L5r-pSqExOJyuUTBH3Ns_itZQh-vcj1OdFpTlZmMx3gNWKq3_zC0wM3Kqll_v3ac4X19CERS5r2iWFQD7t2ndmc9AxLedAoE00JJc7x3tQ_IU9J6F2JputN-ZEq5E0Sr8xMLkD8CDyK5aTmr05k1RyGAUG2XSlbBjxjkULb-kRc4B1wQ/w640-h108/Capture.JPG" width="640" /></a></div><p></p>
<p class="MsoListParagraphCxSpMiddle" style="line-height: normal; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;"><br /></p>
<p class="MsoListParagraphCxSpMiddle" style="line-height: normal; mso-add-space: auto; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;"><span style="color: black; font-family: "Times New Roman",serif; font-size: 13.5pt; mso-fareast-font-family: "Times New Roman";"><b>Below is the Groovy Script:</b><o:p></o:p></span></p>
<span style="font-family: "Times New Roman", serif; font-size: 13.5pt;"><o:p> </o:p></span><span style="font-family: Times New Roman, serif;">import java.util.Properties;</span><br /><span style="font-family: "Times New Roman", serif;">import java.io.InputStream;</span><br /><span style="font-family: "Times New Roman", serif;">import org.jdom.input.SAXBuilder;</span><br /><span style="font-family: "Times New Roman", serif;">import org.jdom.Document;</span><br /><span style="font-family: "Times New Roman", serif;">import org.jdom.Element;</span><br /><span style="font-family: "Times New Roman", serif;">import org.jdom.Namespace;</span><br /><span style="font-family: "Times New Roman", serif;">import org.jdom.xpath.XPath;</span><br /><span style="font-family: "Times New Roman", serif;">import org.jdom.output.XMLOutputter;</span><br /><span style="font-family: "Times New Roman", serif;">import com.boomi.execution.ExecutionUtil;</span><br /><span style="font-family: Times New Roman, serif;"><br /></span><span style="font-family: "Times New Roman", serif;">InputStream is=null;</span><br /><span style="font-family: "Times New Roman", serif;">Properties props=null;</span><br /><span style="font-family: Times New Roman, serif;"><br /></span><span style="font-family: "Times New Roman", serif;">def getItems = { xml,tagname -> xml.'**'.findAll{it.name() == tagname} }</span><br /><span style="font-family: Times New Roman, serif;"><br /></span><span style="font-family: "Times New Roman", serif;">document1XML = new XmlSlurper().parse(dataContext.getStream(0));</span><br /><span style="font-family: Times New Roman, serif;"><br /></span><span style="font-family: "Times New Roman", serif;">props = dataContext.getProperties(0);</span><br /><span style="font-family: Times New Roman, serif;"><br /></span><span style="font-family: "Times New Roman", serif;">//For each document</span><br /><span style="font-family: Times New Roman, serif;"><br /></span><span style="font-family: "Times New Roman", serif;">for( int i = 1; i < dataContext.getDataCount(); i++ ) {</span><br /><span style="font-family: Times New Roman, serif;"><br /></span><span style="font-family: "Times New Roman", serif;"> document2XML = new XmlSlurper().parse(dataContext.getStream(i));</span><br /><span style="font-family: Times New Roman, serif;"><br /></span><span style="font-family: "Times New Roman", serif;">//Extract the node using above filer and insert in base document</span><br /><span style="font-family: Times New Roman, serif;"><br /></span><span style="font-family: "Times New Roman", serif;"> document1XML.Book.children().each {</span><br /><span style="font-family: Times New Roman, serif;"><br /></span><span style="font-family: "Times New Roman", serif;"> tag-></span><br /><span style="font-family: Times New Roman, serif;"><br /></span><span style="font-family: "Times New Roman", serif;"> getItems(document2XML,tag.name()).collect{ tag.appendNode(it.children())}</span><br /><span style="font-family: "Times New Roman", serif;"> }</span><br /><span style="font-family: "Times New Roman", serif;">}</span><br /><span style="font-family: Times New Roman, serif;"><br /></span><span style="font-family: "Times New Roman", serif;">//Seriealize it and assigned it to Boomi Document Context</span><br /><span style="font-family: Times New Roman, serif;"><br /></span><span style="font-family: "Times New Roman", serif;">mergedxmlStr=groovy.xml.XmlUtil.serialize(document1XML);</span><br /><span style="font-family: Times New Roman, serif;"><br /></span><span style="font-family: "Times New Roman", serif;">is = new ByteArrayInputStream(mergedxmlStr.getBytes("UTF-8"));</span><br /><span style="font-family: Times New Roman, serif;"><br /></span><p class="MsoListParagraphCxSpMiddle" style="line-height: normal; text-align: left;"><span style="font-family: Times New Roman, serif;">dataContext.storeStream(is, props);<br /></span></p><br /><p></p>Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-58828131235983731712022-09-04T00:48:00.004+05:302022-09-04T00:48:34.648+05:30Quick Guide - Docker/Container/Container Images/Registry <p>In this post we will be listing and exploring the commands and basic approach to get started build container images with Docker.</p><p>I am using Google Cloud Shell for this blog but you just installed Docker on your local machine. </p><p><i><u>Lets go through each keyword in brief:</u></i></p><p><b>Container Images:</b> Container image is the package of the actual code and all its dependencies. This makes it easy to be executed on any environments/OS etc. </p><p>Example, if you have a nodeJs file say app.js. In order to run the app.js, you will need node version to be installed on the machine. Similarly, in case of a container Image, it will first have a node installation -> your code i.e. app.js in a single package. </p><p><b>Container: </b>Container Images become container when they are running on the platform example, Docker/Kubernetes etc. </p><p><b>Registry: </b>Once you build the image on the local machine, you can check in the image in to Container Registry. This allow it to be pulled from multiple platforms and make it sharable. </p><p><i><span style="font-size: x-small;">Ref in case you need more info: <a href="https://www.docker.com/resources/what-container/">https://www.docker.com/resources/what-container/</a></span></i></p><p><b>Below are some frequently used docker commands:</b></p><p><b><google-sheets-html-origin></google-sheets-html-origin></b></p><table border="1" cellpadding="0" cellspacing="0" dir="ltr" style="border-collapse: collapse; border: none; font-family: Arial; font-size: 10pt; table-layout: fixed; width: 0px;" xmlns="http://www.w3.org/1999/xhtml"><colgroup><col width="268"></col><col width="373"></col></colgroup><tbody><tr style="height: 21px;"><td data-sheets-value="{"1":2,"2":"docker run hello-world"}" style="border: 1px solid rgb(204, 204, 204); overflow: hidden; padding: 2px 3px; vertical-align: bottom;">docker run hello-world</td><td data-sheets-value="{"1":2,"2":"To test if the docker installation is proper."}" style="border: 1px solid rgb(204, 204, 204); overflow-wrap: break-word; overflow: hidden; padding: 2px 3px; vertical-align: bottom; white-space: normal;">To test if the docker installation is proper.</td></tr><tr style="height: 21px;"><td data-sheets-value="{"1":2,"2":"docker images"}" style="border: 1px solid rgb(204, 204, 204); overflow: hidden; padding: 2px 3px; vertical-align: bottom;">docker images</td><td data-sheets-value="{"1":2,"2":"List all the images on the local machine"}" style="border: 1px solid rgb(204, 204, 204); overflow-wrap: break-word; overflow: hidden; padding: 2px 3px; vertical-align: bottom; white-space: normal;">List all the images on the local machine</td></tr><tr style="height: 21px;"><td data-sheets-value="{"1":2,"2":"docker ps"}" style="border: 1px solid rgb(204, 204, 204); overflow: hidden; padding: 2px 3px; vertical-align: bottom;">docker ps</td><td data-sheets-value="{"1":2,"2":"List all the container processes running"}" style="border: 1px solid rgb(204, 204, 204); overflow-wrap: break-word; overflow: hidden; padding: 2px 3px; vertical-align: bottom; white-space: normal;">List all the container processes running</td></tr><tr style="height: 21px;"><td data-sheets-value="{"1":2,"2":"docker ps -a"}" style="border: 1px solid rgb(204, 204, 204); overflow: hidden; padding: 2px 3px; vertical-align: bottom;">docker ps -a</td><td data-sheets-value="{"1":2,"2":"List all the past executed container processes"}" style="border: 1px solid rgb(204, 204, 204); overflow-wrap: break-word; overflow: hidden; padding: 2px 3px; vertical-align: bottom; white-space: normal;">List all the past executed container processes</td></tr><tr style="height: 21px;"><td data-sheets-value="{"1":2,"2":"docker build "}" style="border: 1px solid rgb(204, 204, 204); overflow: hidden; padding: 2px 3px; vertical-align: bottom;">docker build</td><td data-sheets-value="{"1":2,"2":"Build the container Image based on the filename \"Dockerfile\"\n\nExample:\ndocker build -t node-app:0.1\n\n-t indicate tag name [TagName]:[Version]"}" style="border: 1px solid rgb(204, 204, 204); overflow-wrap: break-word; overflow: hidden; padding: 2px 3px; vertical-align: bottom; white-space: normal;">Build the container Image based on the filename "Dockerfile"<br /><br />Example:<br />docker build -t node-app:0.1<br /><br />-t indicate tag name [TagName]:[Version]</td></tr><tr style="height: 21px;"><td data-sheets-value="{"1":2,"2":"docker run"}" style="border: 1px solid rgb(204, 204, 204); overflow: hidden; padding: 2px 3px; vertical-align: bottom;">docker run</td><td data-sheets-hyperlinkruns="{"1":391,"2":"http://localhost:4000/"}{"1":412}{"1":451,"2":"http://localhost/"}{"1":470}" data-sheets-textstyleruns="{"1":0}{"1":99,"2":{"5":1}}{"1":150}{"1":152,"2":{"5":1}}{"1":159}{"1":391,"2":{"2":{"1":2,"2":0}}}{"1":401,"2":{"2":{"1":2,"2":1136076},"9":1}}{"1":422}{"1":451,"2":{"2":{"1":2,"2":0}}}{"1":461,"2":{"2":{"1":2,"2":1136076},"9":1}}{"1":482}" data-sheets-value="{"1":2,"2":"Run the container image.\nExample:\n\nIf you have NodeJS listener you can run it using below command\n\ndocker run -p 4000:80 --name my-app -d node-app:0.1\n\n4000:80 -> Indicate the mapping of the port between the local machine and the docker container.\nIn this case the Node will run on default port 80. In order to call the listener from the local machine, we are mapping 4000 -> 80\n\nThus our URL will be http://localhost:4000 which will call Container listener on http://localhost:80\n\n--name is to name the container\n\n-d is to run the container in background"}" style="border: 1px solid rgb(204, 204, 204); overflow-wrap: break-word; overflow: hidden; padding: 2px 3px; vertical-align: bottom; white-space: normal;"><span style="font-family: Arial; font-size: 10pt; font-style: normal;">Run the container image.<br />Example:<br /><br />If you have NodeJS listener you can run it using below command<br /><br /></span><span style="font-family: Arial; font-size: 10pt; font-style: normal; font-weight: bold;">docker run -p 4000:80 --name my-app -d node-app:0.1</span><span style="font-family: Arial; font-size: 10pt; font-style: normal;"><br /><br /></span><span style="font-family: Arial; font-size: 10pt; font-style: normal; font-weight: bold;">4000:80</span><span style="font-family: Arial; font-size: 10pt; font-style: normal;"> -> Indicate the mapping of the port between the local machine and the docker container.<br />In this case the Node will run on default port 80. In order to call the listener from the local machine, we are mapping 4000 -> 80<br /><br />Thus our UR</span><span style="font-family: Arial; font-size: 10pt; font-style: normal;"><a class="in-cell-link" href="http://localhost:4000/" target="_blank">L will be </a></span><span style="color: #1155cc; font-family: Arial; font-size: 10pt; font-style: normal; text-decoration-skip-ink: none; text-decoration: underline;"><a class="in-cell-link" href="http://localhost:4000/" target="_blank">http://loca</a></span><span style="color: #1155cc; font-family: Arial; font-size: 10pt; font-style: normal; text-decoration-skip-ink: none; text-decoration: underline;">lhost:4000</span><span style="font-family: Arial; font-size: 10pt; font-style: normal;"> which will call Container li</span><span style="font-family: Arial; font-size: 10pt; font-style: normal;"><a class="in-cell-link" href="http://localhost/" target="_blank">stener on </a></span><span style="color: #1155cc; font-family: Arial; font-size: 10pt; font-style: normal; text-decoration-skip-ink: none; text-decoration: underline;"><a class="in-cell-link" href="http://localhost/" target="_blank">http://lo</a></span><span style="color: #1155cc; font-family: Arial; font-size: 10pt; font-style: normal; text-decoration-skip-ink: none; text-decoration: underline;">calhost:80<br /><br /></span><span style="font-family: Arial; font-size: 10pt; font-style: normal;">--name is to name the container<br /><br />-d is to run the container in background</span></td></tr><tr style="height: 21px;"><td data-sheets-value="{"1":2,"2":"docker ps"}" style="border: 1px solid rgb(204, 204, 204); overflow: hidden; padding: 2px 3px; vertical-align: bottom;">docker ps</td><td data-sheets-value="{"1":2,"2":"You can use it to also get the container ID"}" style="border: 1px solid rgb(204, 204, 204); overflow-wrap: break-word; overflow: hidden; padding: 2px 3px; vertical-align: bottom; white-space: normal;">You can use it to also get the container ID</td></tr><tr style="height: 21px;"><td data-sheets-value="{"1":2,"2":"docker logs [containerID]"}" style="border: 1px solid rgb(204, 204, 204); overflow: hidden; padding: 2px 3px; vertical-align: bottom;">docker logs [containerID]</td><td data-sheets-value="{"1":2,"2":"To get the container logs \ndocker logs []"}" style="border: 1px solid rgb(204, 204, 204); overflow-wrap: break-word; overflow: hidden; padding: 2px 3px; vertical-align: bottom; white-space: normal;">To get the container logs<br />docker logs []</td></tr><tr style="height: 21px;"><td data-sheets-value="{"1":2,"2":"docker logs -f [container_id]"}" style="border: 1px solid rgb(204, 204, 204); overflow: hidden; padding: 2px 3px; vertical-align: bottom;">docker logs -f [container_id]</td><td data-sheets-value="{"1":2,"2":"You can use -f in case you want to tail on the container logs"}" style="border: 1px solid rgb(204, 204, 204); overflow-wrap: break-word; overflow: hidden; padding: 2px 3px; vertical-align: bottom; white-space: normal;">You can use -f in case you want to tail on the container logs</td></tr><tr style="height: 21px;"><td data-sheets-value="{"1":2,"2":"docker exec -it [container_id] bash"}" style="border: 1px solid rgb(204, 204, 204); overflow: hidden; padding: 2px 3px; vertical-align: bottom;">docker exec -it [container_id] bash</td><td data-sheets-value="{"1":2,"2":"exec command can be used if you want to SSH in the docker container.\n\nThis will allow you to actually explore the files with the container instances"}" style="border: 1px solid rgb(204, 204, 204); overflow-wrap: break-word; overflow: hidden; padding: 2px 3px; vertical-align: bottom; white-space: normal;">exec command can be used if you want to SSH in the docker container.<br /><br />This will allow you to actually explore the files with the container instances</td></tr><tr style="height: 21px;"><td data-sheets-value="{"1":2,"2":"docker inspect [container_id]"}" style="border: 1px solid rgb(204, 204, 204); overflow: hidden; padding: 2px 3px; vertical-align: bottom;">docker inspect [container_id]</td><td data-sheets-value="{"1":2,"2":"Container's metadata in Docker by using Docker inspect\n\nThis will return a JSON formated output"}" style="border: 1px solid rgb(204, 204, 204); overflow-wrap: break-word; overflow: hidden; padding: 2px 3px; vertical-align: bottom; white-space: normal;">Container's metadata in Docker by using Docker inspect<br /><br />This will return a JSON formated output</td></tr><tr style="height: 21px;"><td data-sheets-value="{"1":2,"2":"docker tag"}" style="border: 1px solid rgb(204, 204, 204); overflow: hidden; padding: 2px 3px; vertical-align: bottom;">docker tag</td><td data-sheets-value="{"1":2,"2":"You can use this to tag existing Image which another Tag. This is needed is you planning to publish images on container registry"}" style="border: 1px solid rgb(204, 204, 204); overflow-wrap: break-word; overflow: hidden; padding: 2px 3px; vertical-align: bottom; white-space: normal;">You can use this to tag existing Image which another Tag. This is needed is you planning to publish images on container registry</td></tr><tr style="height: 21px;"><td colspan="2" data-sheets-textstyleruns="{"1":0}{"1":330,"2":{"5":1}}{"1":364}{"1":371,"2":{"5":1}}" data-sheets-value="{"1":2,"2":"Steps to publish images on GCP registry\n\n1. You will need to tag an image with below format [hostname]/[project-id]/[image]:[tag]\nFor gcr:\n\n[hostname]= gcr.io\n[project-id]= your project's ID\n[image]= your image name\n[tag]= any string tag of your choice. If unspecified, it defaults to \"latest\".\n\nExample:\n\ndocker tag node-app:0.2 gcr.io/[project-id]/node-app:0.2\n\ndocker push gcr.io/[project-id]/node-app:0.2\n"}" rowspan="1" style="border: 1px solid rgb(204, 204, 204); overflow: hidden; padding: 2px 3px; vertical-align: bottom;"><span style="font-family: Arial; font-size: 10pt; font-style: normal;">Steps to publish images on GCP registry<br /><br />1. You will need to tag an image with below format [hostname]/[project-id]/[image]:[tag]<br />For gcr:<br /><br />[hostname]= gcr.io<br />[project-id]= your project's ID<br />[image]= your image name<br />[tag]= any string tag of your choice. If unspecified, it defaults to "latest".<br /><br />Example:<br /><br />docker tag node-app:0.2 </span><span style="font-family: Arial; font-size: 10pt; font-style: normal; font-weight: bold;">gcr.io/[project-id]/node-app:0.2<br /><br /></span><span style="font-family: Arial; font-size: 10pt; font-style: normal;">docker </span><span style="font-family: Arial; font-size: 10pt; font-style: normal; font-weight: bold;">push gcr.io/[project-id]/node-app:0.2<br /></span></td></tr><tr style="height: 21px;"><td data-sheets-value="{"1":2,"2":"docker stop [Container ID]"}" style="border: 1px solid rgb(204, 204, 204); overflow: hidden; padding: 2px 3px; vertical-align: bottom;">docker stop [Container ID]</td><td data-sheets-value="{"1":2,"2":"To stop the running container"}" style="border: 1px solid rgb(204, 204, 204); overflow-wrap: break-word; overflow: hidden; padding: 2px 3px; vertical-align: bottom; white-space: normal;">To stop the running container</td></tr><tr style="height: 21px;"><td data-sheets-value="{"1":2,"2":"docker rm [container ID]"}" style="border: 1px solid rgb(204, 204, 204); overflow: hidden; padding: 2px 3px; vertical-align: bottom;">docker rm [container ID]</td><td data-sheets-value="{"1":2,"2":"To remove the container instance"}" style="border: 1px solid rgb(204, 204, 204); overflow-wrap: break-word; overflow: hidden; padding: 2px 3px; vertical-align: bottom; white-space: normal;">To remove the container instance</td></tr><tr style="height: 21px;"><td data-sheets-value="{"1":2,"2":"docker rmi [image id]"}" style="border: 1px solid rgb(204, 204, 204); overflow: hidden; padding: 2px 3px; vertical-align: bottom;">docker rmi [image id]</td><td data-sheets-value="{"1":2,"2":"To remove the images.\nImage ID can be fetched using docker images"}" style="border: 1px solid rgb(204, 204, 204); overflow-wrap: break-word; overflow: hidden; padding: 2px 3px; vertical-align: bottom; white-space: normal;">To remove the images.<br />Image ID can be fetched using docker images</td></tr><tr style="height: 21px;"><td data-sheets-value="{"1":2,"2":"docker pull"}" style="border: 1px solid rgb(204, 204, 204); overflow: hidden; padding: 2px 3px; vertical-align: bottom;">docker pull</td><td data-sheets-hyperlinkruns="{"1":177,"2":"http://localhost:4000/"}{"1":198}" data-sheets-textstyleruns="{"1":0}{"1":181,"2":{"2":{"1":2,"2":1136076},"9":1}}" data-sheets-value="{"1":2,"2":"To pull the image from the remote directory\n\nexample to pull and run:\n\ndocker pull gcr.io/[project-id]/node-app:0.2\r\ndocker run -p 4000:80 -d gcr.io/[project-id]/node-app:0.2\r\ncurl http://localhost:4000"}" style="border: 1px solid rgb(204, 204, 204); overflow-wrap: break-word; overflow: hidden; padding: 2px 3px; vertical-align: bottom; white-space: normal;"><span style="font-family: Arial; font-size: 10pt; font-style: normal;">To pull the image from the remote directory<br /><br />example to pull and run:<br /><br />docker pull gcr.io/[project-id]/node-app:0.2<br />docker run -p 4000:80 -d gcr.io/[project-id]/node-app:0.2<br />c</span><span style="font-family: Arial; font-size: 10pt; font-style: normal;"><a class="in-cell-link" href="http://localhost:4000/" target="_blank">url </a></span><span style="color: #1155cc; font-family: Arial; font-size: 10pt; font-style: normal; text-decoration-skip-ink: none; text-decoration: underline;"><a class="in-cell-link" href="http://localhost:4000/" target="_blank">http://localhost:</a></span><span style="color: #1155cc; font-family: Arial; font-size: 10pt; font-style: normal; text-decoration-skip-ink: none; text-decoration: underline;">4000</span></td></tr></tbody></table><p><br /></p><p>If you want to perform an hands on steps, you can refer below document. </p><p><a href="https://www.cloudskillsboost.google/focuses/1029?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&parent=catalog&search_id=17839779"><span style="font-size: x-small;">https://www.cloudskillsboost.google/focuses/1029?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&parent=catalog&search_id=17839779</span></a></p><p><br /></p><p><br /></p><p><br /></p>Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-4853431264200300532022-08-22T03:36:00.005+05:302022-08-22T04:00:27.435+05:30Getting started with Data Engineering - Google Cloud Notes<p>Data engineering at high level is to design, build, monitor and secure the data processing pipelines. </p><p>Data pipelines mainly perform ETL (Extract, Transform, Load) jobs, which transform the source data (structure/un-structure) to a data warehouse in a meaningful way, which can than be used for analytics or machine learning models. </p><p>Data processing can be in form of batch or streaming. On GCP, we can use Dataproc (Apache Hadoop, Spark etc.) for Batch processing and Dataflow (Apache Beam programing) for Batch & Streaming. </p><p>Batch Pipeline:</p><p style="text-align: left;">Processing the data in the form of batches. Examples: A nightly Job to extract sales transactions for analysis. </p><p>Streaming Pipeline:</p><p style="text-align: left;">Processing continues stream of data. Examples: IoT events, payment processing events, logs. Streaming processing is used when you need a near real-time outcome or decision making. </p><p><br /></p><p>To get started on Data engineering, would recommend you to go through below courses or labs. </p><p></p><ul style="text-align: left;"><li><a href="https://www.cloudskillsboost.google/course_templates/54">Modernizing Data Lakes and Data Warehouses with Google Cloud</a></li><li><a href="https://www.cloudskillsboost.google/course_templates/218">Serverless Data Processing with Dataflow: Foundations</a></li><li><a href="https://www.cloudskillsboost.google/course_templates/229" target="_blank">Serverless Data Processing with Dataflow: Develop Pipelines</a></li></ul><p></p><p><br /></p><p>I personally feel if we are planning to build new data processing pipelines, or trying to learn data engineering than we should consider Dataflow over traditional Hadoop/Spark jobs. The reason been much of the infra setup complexity is abstracted as part of Google's Managed service offering which make the learning curve bit smoother. </p><p><i><br /></i></p><p><i>Will continue posting further Dataflow and Data engineering learning exercises.....</i></p><p>-----------------------</p><p><i>If you need a short desc on Google Cloud products, you can refer below cheat sheet:</i></p><p><a href="https://googlecloudcheatsheet.withgoogle.com/" target="_blank"><i>Google Cloud Cheat Sheet</i></a></p><p><br /></p><p><br /></p>Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-10536191045473422312022-07-31T06:42:00.008+05:302022-07-31T06:42:59.104+05:30Introduction to Google Apigee X<p><br /></p><p> <b>Google Apigee X</b> is the API management platform which allow to develop and manager experience API's. </p><p>This APIs can be made available to multiple clients with different security policies. </p><p>Example, a Mobile Developer might need a JWT access token validation policy vs a traditional system might need same API with HTTP Basic policy. </p><p>As an API management platform, it allow several security policies like SQL injections, JSON Threat Protection etc. </p><p>Google Apigee X is full managed API management platform. It consist of a management plane and runtime plane. You can also implement Hybrid model i.e. Cloud & On-Premise runtime instances. </p><p><br /></p><p><u>You can refer, below link for high level architecture:</u></p><p><span id="docs-internal-guid-817f98e8-7fff-40c5-867a-3a97b20ae7dd"><span style="background-color: white; color: #202124; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><span style="border: none; display: inline-block; height: 233px; overflow: hidden; width: 624px;"><img height="233" src="https://lh3.googleusercontent.com/ccPZOOLTtrCBwRZ8oXqhdlJ3c1q76Ct1eBd-chR21NTv3AazJFdFNDHlD14rxxzGZiZa5LZRhXi2udtiqo1MzCIShra0r78yYI1VDssKA7FfNladpEoBk2hcT7BR-UcXolbVUPKifVydRUB4DEfAZOE" style="margin-left: 0px; margin-top: 0px;" width="624" /></span></span></span></p><p><a href="https://cloud.google.com/apigee/docs/api-platform/architecture/overview">https://cloud.google.com/apigee/docs/api-platform/architecture/overview</a></p><p><br /></p><p>For a hands on experience and details overview, would recommend completing below course and labs:</p><p><br /></p><p><u><b>Course:</b></u></p><p></p><ul style="text-align: left;"><li><b>API Design and Fundamentals of Google Cloud's Apigee API Platform</b></li></ul><p></p><p><span> </span><span> </span><span> </span><a href="https://www.cloudskillsboost.google/course_templates/254">https://www.cloudskillsboost.google/course_templates/254</a></p><p><b><u>Labs:</u></b></p><p></p><ul style="text-align: left;"><li><b>Deploy and Manage Apigee X</b></li></ul><p></p><p></p><div><span> </span><span> </span><span> </span><a href="https://www.cloudskillsboost.google/quests/181">https://www.cloudskillsboost.google/quests/181</a></div><div><br /></div><div><ul style="text-align: left;"><li><b>Develop and Secure APIs with Apigee X</b></li></ul></div><div><span> </span><span> </span><span> </span><a href="https://www.cloudskillsboost.google/quests/194">https://www.cloudskillsboost.google/quests/194</a></div><p></p>Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-7292699903190319992021-02-12T07:48:00.001+05:302021-02-12T07:48:24.903+05:30API Design First approach: Implementing quick mock API's using swagger hub and postman<p>Consider a scenario, where you as a API developer been ask to to implement the API's for a new mobile/web application team. The requirement is to implement a application to allow users to manage orders.</p><p> Mobile APP -> API infra->Backend enterprise system</p><p>While implement a full working API might take days, you also do not want the application team to wait till the API's are ready. Thus, in most case we end up implementing the mock API's with sample scenarios.</p><p>In this post, we will define an API using swagger openapi 3.0 yaml specification in swaggerhub. We will then use the definition to implement mock API's in postman.</p><p>Even though postman provide option to define the API's, i liked swagger hub much better, it allow you to view your API on right side. </p><p>For this post I implemented API definition in swagger, please find the below link.</p><p><a href="https://app.swaggerhub.com/apis-docs/lopesrohan1988/ordermanagement/1.0.0">https://app.swaggerhub.com/apis-docs/lopesrohan1988/ordermanagement/1.0.0</a></p><p>API Definition can be found at below link:</p><p><a href="https://github.com/lopesrohan1988/CloudNaiveApp/blob/main/API/ApiSpec.yaml">https://github.com/lopesrohan1988/CloudNaiveApp/blob/main/API/ApiSpec.yaml</a></p><p><br /></p><p>Once you have the API definition agreed with all the stack holders, next step is to implement mock API's. </p><p>You can follow below steps to setup mock api's:</p><p>1. Login to Postman.</p><p>2. Create API by copying the API definition from SwaggerHUB</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-1NheAiAE08I/YCXfJYQROoI/AAAAAAABTnw/S8QCgkBNhikWyWfQh3KdPOAygzjEHr-tgCLcBGAsYHQ/s751/Capture.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="204" data-original-width="751" height="109" src="https://1.bp.blogspot.com/-1NheAiAE08I/YCXfJYQROoI/AAAAAAABTnw/S8QCgkBNhikWyWfQh3KdPOAygzjEHr-tgCLcBGAsYHQ/w400-h109/Capture.PNG" width="400" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><br /></div>3. Click on Generate Collection button and select API Mocking option<div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-fXBTh83PZMI/YCXfpFIkvNI/AAAAAAABTn4/NShb6GdlqmkyXz1STkg-wV3vr8ijzpY1wCLcBGAsYHQ/s694/Capture.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="432" data-original-width="694" height="249" src="https://1.bp.blogspot.com/-fXBTh83PZMI/YCXfpFIkvNI/AAAAAAABTn4/NShb6GdlqmkyXz1STkg-wV3vr8ijzpY1wCLcBGAsYHQ/w400-h249/Capture.PNG" width="400" /></a></div><br /><div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-22OGUDoviPk/YCXf6D3UUwI/AAAAAAABToA/H6kNiLObv3AYVut7TH1bAPM_1z5nlMTkgCLcBGAsYHQ/s348/Capture.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="346" data-original-width="348" src="https://1.bp.blogspot.com/-22OGUDoviPk/YCXf6D3UUwI/AAAAAAABToA/H6kNiLObv3AYVut7TH1bAPM_1z5nlMTkgCLcBGAsYHQ/s320/Capture.PNG" width="320" /></a></div><br /><br /><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p>4. Now click on collections, and under your collection you will find the mock auto generated responses based on the API definition.</p><p>You can modify the responses or even add new responses based on the input parameter.</p><p>For example, I added 2 success response for order # 100 and 200 and 1 not found response for order # 300.</p><p>Example:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-e7eFceHtnMw/YCXi5Cyr2GI/AAAAAAABToM/Cqd3dD18lr8k_b1OInNzsePSFL5pcoUdgCLcBGAsYHQ/s1279/Capture.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="525" data-original-width="1279" src="https://1.bp.blogspot.com/-e7eFceHtnMw/YCXi5Cyr2GI/AAAAAAABToM/Cqd3dD18lr8k_b1OInNzsePSFL5pcoUdgCLcBGAsYHQ/s320/Capture.PNG" width="320" /></a></div><br /><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p>5. Once the Mock server is setup, you can copy the mock server URL and update the swagger definition on SwaggerHub. This will allow you to test your API's from SwaggerUI with mock server as a backend.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-pYmNWv2gFIA/YCXkgIFdHcI/AAAAAAABToY/TS42v1kj9ucgXyfH0-HxIQLWtHGFY53SgCLcBGAsYHQ/s1141/Capture.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="597" data-original-width="1141" src="https://1.bp.blogspot.com/-pYmNWv2gFIA/YCXkgIFdHcI/AAAAAAABToY/TS42v1kj9ucgXyfH0-HxIQLWtHGFY53SgCLcBGAsYHQ/s320/Capture.PNG" width="320" /></a></div><br /><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p>Also, since the mock collection is maintained in postman, it is very easy to change or added new mock responses. There are lot of API design products, but I found it very easy and quick to get it implemented using <b>postman </b>& <b>swaggerhub</b>.</p><p><br /></p><p><a href="https://www.postman.com/">https://www.postman.com/</a></p><p><a href="https://app.swaggerhub.com/">https://app.swaggerhub.com/</a></p><p><br /></p><p><br /></p></div>Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-13729439390211396272020-05-24T22:51:00.000+05:302020-05-24T22:51:27.267+05:30Using Google Cloud Function to save JSON payload from Google Storage Event to Google Bigquery<u>As part of this article we will learn below:</u><div><ul style="text-align: left;"><li>Create the cloud storage bucket</li><li>Use sample hierarchical JSON payload to auto create bigquery table</li><li>Create google cloud function to listen to the object creation event on storage bucket and Fetch the JSON payload from the file and load in to Bigquery</li></ul><div>*We will be using Python for this activity.</div><div><br /></div><div><b><u>1. Create the cloud storage bucket</u></b></div></div><div><b><br /></b></div><div>This is the simple task, you can either create the bucket using the console or gsutil command. For this exercise, we will use console. We will ignore some parameters like regional, bucket type, as we will not be using this bucket for large storage.</div><div><br /></div><div>Go to cloud storage -> Click on Create Bucket</div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-kPepbqOo_Yc/XsqlA6eQhrI/AAAAAAABHLY/_tfz_iVOHYY0YmmhrMqU46OshxxYlBiEgCK4BGAsYHg/Capture.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="140" data-original-width="899" src="https://1.bp.blogspot.com/-kPepbqOo_Yc/XsqlA6eQhrI/AAAAAAABHLY/_tfz_iVOHYY0YmmhrMqU46OshxxYlBiEgCK4BGAsYHg/s320/Capture.PNG" width="320" /></a></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div>Give some unique name, you can use your Google Project ID+some name and click on create bucket.</div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-L2PT7ycCGQU/XsqlYOpDkcI/AAAAAAABHLs/Wt5g6Iiup_wFv5mf8ng55BKm244PaPnPQCK4BGAsYHg/Capture.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="544" data-original-width="932" src="https://1.bp.blogspot.com/-L2PT7ycCGQU/XsqlYOpDkcI/AAAAAAABHLs/Wt5g6Iiup_wFv5mf8ng55BKm244PaPnPQCK4BGAsYHg/s320/Capture.PNG" width="320" /></a></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><u><b>2. Use sample hierarchical JSON payload to auto create bigquery table</b></u></div><div><u><b><br /></b></u></div><div>One of the quick way to create the Bigquery table to save the JSON payload with array is to first create the sample JSON payload and use it to create the bigquery table.</div><div><br /></div><div><u>For Example:</u> </div><div><br /></div><div>You are expecting the Sales Order data in the JSON payload like below i.e. OrderNo, Suppliername and array of the items in the Sales Order.</div><div><br /></div><div><pre style="text-align: left;"><i>import json<br />data = {<br /> "orderno":"SO1010",<br /> "suppliername":"DUMMY SUPP",<br /> <br /> "parts": [<br /> {<br /> "partnumber": "PA1010",<br /> "partdesc": "PIPE",<br /> "qty": 10<br /> },<br /> {<br /> "partnumber": "PA1010",<br /> "partdesc": "PIPE",<br /> "qty": 10<br /> },<br /> ],<br /> <br />}</i></pre></div><div>Instead of manually creating the BigQuery table, which can be little boring, we can use below approach.</div><div><br /></div><div><ul style="text-align: left;"><li>Convert the JSON to the DELIMITED format - Bigquery need the data in DELIMITED format only</li></ul><pre style="text-align: left;">You can use <i>json.dumps(data)</i> for the same.</pre></div><pre style="text-align: left;">print(json.dumps(data))</pre><div><pre style="background-color: white; border: none; box-sizing: unset; color: var(--jp-content-font-color1); font-family: var(--jp-code-font-family); font-size: var(--jp-code-font-size); line-height: var(--jp-code-line-height); margin-bottom: 0px; margin-top: 0px; overflow-wrap: break-word; overflow: auto; padding: 0px; text-align: left; white-space: pre-wrap; word-break: break-all;">{"orderno": "SO1010", "suppliername": "DUMMY SUPP", "parts": [{"partnumber": "PA1010", "partdesc": "PIPE", "qty": 10}, {"partnumber": "PA1010", "partdesc": "PIPE", "qty": 10}]}</pre></div><div><br /></div><div>Open the Google Cloud Shell, Open editor and create the file with above json payload say <b>Blog.json</b>.</div><div><a href="https://1.bp.blogspot.com/-kCaBi4RfTNQ/XsqoVaT5G6I/AAAAAAABHMM/4yAM6WNiLVQk6o0R0R1RaQOF4UsB9i1-wCK4BGAsYHg/Capture.PNG" imageanchor="1" style="clear: left; display: inline !important; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="48" data-original-width="236" src="https://1.bp.blogspot.com/-kCaBi4RfTNQ/XsqoVaT5G6I/AAAAAAABHMM/4yAM6WNiLVQk6o0R0R1RaQOF4UsB9i1-wCK4BGAsYHg/Capture.PNG" /></a></div><div><br /></div><div>Use below command to create the table in Bigquery:</div><div><br /></div><pre style="text-align: left;">bq load --autodetect --replace --source_format=NEWLINE_DELIMITED_JSON \n</pre><pre style="text-align: left;">mydataset.BLOGDEMOTABLE ./Blog.json</pre><pre style="text-align: left;"><br /></pre><div>where BLOGDEMOTABLE -> Table Name</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-UjPCBkiFkuE/XsqpDNPjOpI/AAAAAAABHMg/VVM1NCRgOSMWHPDYkmxYZKIm2_ni79nVwCK4BGAsYHg/Capture.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="122" data-original-width="1354" src="https://1.bp.blogspot.com/-UjPCBkiFkuE/XsqpDNPjOpI/AAAAAAABHMg/VVM1NCRgOSMWHPDYkmxYZKIm2_ni79nVwCK4BGAsYHg/s320/Capture.PNG" width="320" /></a></div><div><br /></div><div><br /></div><div><br /></div><div>Now if you check in Bigquery you will find the new Table Name BLOGDEMOTABLE. Note the column of type REPEATED to save the array.</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-PFO_1afybRs/XsqpuitebGI/AAAAAAABHM8/7nL1f5iL6VE3wDD1el7g6w3Ax-sja-H3wCK4BGAsYHg/Capture.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="367" data-original-width="366" height="320" src="https://1.bp.blogspot.com/-PFO_1afybRs/XsqpuitebGI/AAAAAAABHM8/7nL1f5iL6VE3wDD1el7g6w3Ax-sja-H3wCK4BGAsYHg/s320/Capture.PNG" /></a></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-yuO1A4UDUVI/Xsqqhq6rOtI/AAAAAAABHNg/wgKq_lvS3ewv5Bhc5eRnLXANFBGxB7M_wCK4BGAsYHg/Capture.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="100" data-original-width="494" src="https://1.bp.blogspot.com/-yuO1A4UDUVI/Xsqqhq6rOtI/AAAAAAABHNg/wgKq_lvS3ewv5Bhc5eRnLXANFBGxB7M_wCK4BGAsYHg/s320/Capture.PNG" width="320" /></a></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><b><u>3. Create google cloud function to listen to the object creation event on storage bucket and Fetch the JSON payload from the file and load in to Bigquery</u></b></div><div><b><u><br /></u></b></div><div><br /></div><div>Create the a new Cloud Function (For AWS Folks it is similar to <b>lambda</b> (Function), and select the input as Google Storage Trigger on Create Event.</div><div><br /></div><div>Select runtime as Python</div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-rRTMZ7-rIGQ/XsqrLnGFVMI/AAAAAAABHOA/qyukteEBRFYW4xtp1eQeWgs4iMpjqRy-gCK4BGAsYHg/Capture.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="382" data-original-width="676" src="https://1.bp.blogspot.com/-rRTMZ7-rIGQ/XsqrLnGFVMI/AAAAAAABHOA/qyukteEBRFYW4xtp1eQeWgs4iMpjqRy-gCK4BGAsYHg/s320/Capture.PNG" width="320" /></a></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div>Note: Since this function will access google cloud storage and google Bigquery Python Client libraries, we need to add this as the dependencies to the cloud function. </div><div>We can add those dependencies in Requirement.txt, this will enable Cloud Function to install the required package while deployment.</div><div><br /></div><div>Example: (Note you might not need storage dependency for this exercise)</div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-sMD4SR6rQHY/XsqsEvGt5vI/AAAAAAABHOc/Flih96be3mgNJl4zpLsemHUXCodKDbiHgCK4BGAsYHg/Capture.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="169" data-original-width="527" src="https://1.bp.blogspot.com/-sMD4SR6rQHY/XsqsEvGt5vI/AAAAAAABHOc/Flih96be3mgNJl4zpLsemHUXCodKDbiHgCK4BGAsYHg/s320/Capture.PNG" width="320" /></a></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div>Now we just have to right the small code to listen to perform below</div><div><ul style="text-align: left;"><li>Listen to the cloud create file event</li><li>Get the file name</li><li>Create Bigquery JobConfig object and call load_table_from_uri function</li></ul><div>Please find the below code.</div></div><div><br /></div><div style="text-align: left;"><div style="background-color: #fffffe; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><pre><span style="color: blue;">from</span> google.cloud <span style="color: blue;">import</span> bigquery<br /><br /><span style="color: blue;">def</span> insertdatatobigquery(event, context):<br /> <span style="color: #a31515;">"""Triggered by a change to a Cloud Storage bucket.<br /></span><span style="color: #a31515;"> Args:<br /></span><span style="color: #a31515;"> event (dict): Event payload.<br /></span><span style="color: #a31515;"> context (google.cloud.functions.Context): Metadata for the event.<br /></span><span style="color: #a31515;"> """<br /></span> <span style="color: blue;">file</span> = event<br /> <span style="color: blue;">print</span>(f<span style="color: #a31515;">"Processing file: {file['name']}."</span>)<br /> client = bigquery.Client()</pre><pre><span> #Dataset name</span><br /> dataset_id = <span style="color: #a31515;">'mydataset'</span><br /> dataset_ref = client.dataset(dataset_id)<br /> job_config = bigquery.LoadJobConfig()<br /> job_config.source_format = bigquery.SourceFormat.NEWLINE_DELIMITED_JSON<br /> </pre><pre><span> #Storage filename</span><br /> uri=<span style="color: #a31515;">"gs://"</span>+event[<span style="color: #a31515;">"bucket"</span>]+<span style="color: #a31515;">'/'</span>+event[<span style="color: #a31515;">"name"</span>]<br /> </pre><pre><span> #This function load the data from JSON file to Bigquery table</span><br /></pre><pre><span> </span>load_job = client.load_table_from_uri(<br /> uri,<br /> dataset_ref.table(<span style="color: #a31515;">"PACKINGSLIPDOCUMENTS"</span>),<br /> location=<span style="color: #a31515;">"US"</span>, <span style="color: green;"># Location must match that of the destination dataset.<br /></span> job_config=job_config,<br /> ) <span style="color: green;"># API request</span><br /> <span style="color: blue;">print</span>(<span style="color: #a31515;">"Starting job {}"</span>.<span style="color: blue;">format</span>(load_job.job_id))<br /> load_job.result() <span style="color: green;"># Waits for table load to complete.</span><br /> <span style="color: blue;">print</span>(<span style="color: #a31515;">"Big Query Job finished."</span>) <br /><br /></pre><pre><br /></pre><br /></div></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div>Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-47751844383048099932020-05-10T00:25:00.007+05:302020-05-10T01:16:10.954+05:30Using Sagemaker to train and serve Tensorflow Model<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<br />
In the exercise we are going to use the Kaggle cats and dogs data. Some of the code is from the training course "TensorFlow in Practice".<br />
<br />
<b><u>Part1:</u></b><br />
<div>
<b><u><br /></u></b>First step towards building the machine learning model is to prepare the dataset. In this notebook we will perform below:<br />
Download the kaggle cat and dog data set <br />
Extract the zip <br />
Upload the data set to Amazon S3 bucket <br />
<br />
<pre style="background-color: #f7f7f7; border-radius: 2px; border: none; box-sizing: border-box; color: #333333; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 0px; white-space: pre-wrap; word-break: break-all;"><span class="c1" style="box-sizing: border-box; color: #408080; font-style: italic;">#Download the kaggle data set</span>
<span class="o" style="box-sizing: border-box; color: #666666;">!</span>wget --no-check-certificate <span class="err" style="border: 1px solid rgb(255, 0, 0); box-sizing: border-box;">\</span>
<span class="n" style="box-sizing: border-box;">https</span><span class="p" style="box-sizing: border-box;">:</span><span class="o" style="box-sizing: border-box; color: #666666;">//</span><span class="n" style="box-sizing: border-box;">storage</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">googleapis</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">com</span><span class="o" style="box-sizing: border-box; color: #666666;">/</span><span class="n" style="box-sizing: border-box;">mledu</span><span class="o" style="box-sizing: border-box; color: #666666;">-</span><span class="n" style="box-sizing: border-box;">datasets</span><span class="o" style="box-sizing: border-box; color: #666666;">/</span><span class="n" style="box-sizing: border-box;">cats_and_dogs_filtered</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">zip</span> \
<span class="o" style="box-sizing: border-box; color: #666666;">-</span><span class="n" style="box-sizing: border-box;">O</span> <span class="o" style="box-sizing: border-box; color: #666666;">./</span><span class="n" style="box-sizing: border-box;">cats_and_dogs_filtered</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">zip</span></pre>
<br />
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;">Extract the zip file to local directory</span><br />
<div>
<span style="font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif;"><span style="font-size: 14px;"><br /></span></span></div>
<div>
<pre style="background-color: #f7f7f7; border-radius: 2px; border: none; box-sizing: border-box; color: #333333; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 0px; white-space: pre-wrap; word-break: break-all;"><span class="kn" style="box-sizing: border-box; color: green; font-weight: bold;">import</span> <span class="nn" style="box-sizing: border-box; color: blue; font-weight: bold;">os</span>
<span class="kn" style="box-sizing: border-box; color: green; font-weight: bold;">import</span> <span class="nn" style="box-sizing: border-box; color: blue; font-weight: bold;">zipfile</span>
<span class="n" style="box-sizing: border-box;">local_zip</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="s1" style="box-sizing: border-box; color: #ba2121;">'./cats_and_dogs_filtered.zip'</span>
<span class="n" style="box-sizing: border-box;">zip_ref</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">zipfile</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">ZipFile</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">local_zip</span><span class="p" style="box-sizing: border-box;">,</span> <span class="s1" style="box-sizing: border-box; color: #ba2121;">'r'</span><span class="p" style="box-sizing: border-box;">)</span>
<span class="n" style="box-sizing: border-box;">zip_ref</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">extractall</span><span class="p" style="box-sizing: border-box;">(</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'./Data'</span><span class="p" style="box-sizing: border-box;">)</span>
<span class="n" style="box-sizing: border-box;">zip_ref</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">close</span><span class="p" style="box-sizing: border-box;">()</span></pre>
</div>
<div>
<br /></div>
<div>
<span class="n" style="box-sizing: border-box; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;">Use</span><span style="background-color: #f7f7f7; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;"> </span><span class="n" style="box-sizing: border-box; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;">Amazon</span><span style="background-color: #f7f7f7; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;"> </span><span class="n" style="box-sizing: border-box; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;">SDK</span><span style="background-color: #f7f7f7; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;"> </span><span class="n" style="box-sizing: border-box; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;">to</span><span style="background-color: #f7f7f7; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;"> </span><span class="n" style="box-sizing: border-box; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;">upload</span><span style="background-color: #f7f7f7; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;"> </span><span class="n" style="box-sizing: border-box; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;">the</span><span style="background-color: #f7f7f7; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;"> </span><span class="n" style="box-sizing: border-box; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;">data</span><span style="background-color: #f7f7f7; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;"> </span><span class="n" style="box-sizing: border-box; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;">to</span><span style="background-color: #f7f7f7; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;"> </span><span class="n" style="box-sizing: border-box; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;">S3</span><span style="background-color: #f7f7f7; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;"> </span><span class="n" style="box-sizing: border-box; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;">bucket</span><span class="o" style="box-sizing: border-box; color: #666666; font-family: monospace; font-size: 14px; white-space: pre-wrap;">.</span><span style="background-color: #f7f7f7; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;"> </span><span class="n" style="box-sizing: border-box; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;">I</span><span style="background-color: #f7f7f7; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;"> </span><span class="n" style="box-sizing: border-box; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;">created</span><span style="background-color: #f7f7f7; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;"> </span><span class="n" style="box-sizing: border-box; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;">the</span><span style="background-color: #f7f7f7; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;"> </span><span class="n" style="box-sizing: border-box; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;">bucket</span><span style="background-color: #f7f7f7; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;"> </span><span class="n" style="box-sizing: border-box; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;">name</span><span style="background-color: #f7f7f7; color: #333333; font-family: monospace; font-size: 14px; white-space: pre-wrap;"> </span><span class="s2" style="box-sizing: border-box; color: #ba2121; font-family: monospace; font-size: 14px; white-space: pre-wrap;">"sagemaker-05may2020842"</span></div>
<div>
<br /></div>
<div>
<pre style="background-color: #f7f7f7; border-radius: 2px; border: none; box-sizing: border-box; color: #333333; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 0px; white-space: pre-wrap; word-break: break-all;"><span class="c1" style="box-sizing: border-box; color: #408080; font-style: italic;">#Copy the data to AWS from Local</span>
<span class="kn" style="box-sizing: border-box; color: green; font-weight: bold;">import</span> <span class="nn" style="box-sizing: border-box; color: blue; font-weight: bold;">boto3</span><span class="o" style="box-sizing: border-box; color: #666666;">,</span> <span class="nn" style="box-sizing: border-box; color: blue; font-weight: bold;">os</span>
<span class="n" style="box-sizing: border-box;">s3</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">boto3</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">resource</span><span class="p" style="box-sizing: border-box;">(</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'s3'</span><span class="p" style="box-sizing: border-box;">)</span>
<span class="o" style="box-sizing: border-box; color: #666666;">!</span>aws s3 cp ./Data/* s3://sagemaker-05may2020842/ --recursive</pre>
</div>
<div>
<br /></div>
<div>
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;"><u><b>Part 2:</b></u></span></div>
<div>
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;"><br /></span></div>
<div>
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;">Next we have prepared the script in the required Sagemaker format to train and create Tensorflow Model. You can copy the entire script from the bottom, but we will explain each block</span></div>
<div>
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;"><br /></span></div>
<div>
<pre style="background-color: #f7f7f7; border-radius: 2px; border: none; box-sizing: border-box; color: #333333; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 0px; white-space: pre-wrap; word-break: break-all;"><span class="n" style="box-sizing: border-box;">In</span> <span class="n" style="box-sizing: border-box;">below</span> <span class="n" style="box-sizing: border-box;">function</span><span class="p" style="box-sizing: border-box;">,</span> <span class="n" style="box-sizing: border-box;">we</span> <span class="n" style="box-sizing: border-box;">create</span> <span class="n" style="box-sizing: border-box;">the</span> <span class="n" style="box-sizing: border-box;">Model</span> <span class="k" style="box-sizing: border-box; color: green; font-weight: bold;">with</span> <span class="mi" style="box-sizing: border-box; color: #666666;">3</span> <span class="n" style="box-sizing: border-box;">Convo</span> <span class="n" style="box-sizing: border-box;">layers</span> <span class="n" style="box-sizing: border-box;">before</span> <span class="n" style="box-sizing: border-box;">flatting</span> <span class="ow" style="box-sizing: border-box; color: #aa22ff; font-weight: bold;">and</span> <span class="n" style="box-sizing: border-box;">passing</span> <span class="n" style="box-sizing: border-box;">it</span> <span class="n" style="box-sizing: border-box;">to</span> <span class="n" style="box-sizing: border-box;">Dense</span> <span class="n" style="box-sizing: border-box;">layer</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span>
<span class="n" style="box-sizing: border-box;">Since</span> <span class="n" style="box-sizing: border-box;">we</span> <span class="n" style="box-sizing: border-box;">have</span> <span class="n" style="box-sizing: border-box;">only</span> <span class="mi" style="box-sizing: border-box; color: #666666;">2</span> <span class="n" style="box-sizing: border-box;">classes</span><span class="p" style="box-sizing: border-box;">,</span> <span class="n" style="box-sizing: border-box;">we</span> <span class="n" style="box-sizing: border-box;">are</span> <span class="n" style="box-sizing: border-box;">using</span> <span class="n" style="box-sizing: border-box;">binary_crossentropy</span> <span class="n" style="box-sizing: border-box;">loss</span> <span class="n" style="box-sizing: border-box;">function</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span> <span class="n" style="box-sizing: border-box;">Also</span> <span class="n" style="box-sizing: border-box;">we</span> <span class="n" style="box-sizing: border-box;">are</span> <span class="n" style="box-sizing: border-box;">using</span>
<span class="n" style="box-sizing: border-box;">RMS</span> <span class="k" style="box-sizing: border-box; color: green; font-weight: bold;">with</span> <span class="n" style="box-sizing: border-box;">learning</span> <span class="n" style="box-sizing: border-box;">rate</span> <span class="n" style="box-sizing: border-box;">of</span> <span class="mf" style="box-sizing: border-box; color: #666666;">0.001</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span> <span class="n" style="box-sizing: border-box;">This</span> <span class="n" style="box-sizing: border-box;">are</span> <span class="n" style="box-sizing: border-box;">experimental</span> <span class="n" style="box-sizing: border-box;">values</span><span class="p" style="box-sizing: border-box;">,</span> <span class="n" style="box-sizing: border-box;">which</span> <span class="n" style="box-sizing: border-box;">can</span> <span class="n" style="box-sizing: border-box;">be</span> <span class="n" style="box-sizing: border-box;">tuned</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span>
<span class="n" style="box-sizing: border-box;">model</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">compile</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">optimizer</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="n" style="box-sizing: border-box;">RMSprop</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">lr</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="mf" style="box-sizing: border-box; color: #666666;">0.001</span><span class="p" style="box-sizing: border-box;">),</span>
<span class="n" style="box-sizing: border-box;">loss</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'binary_crossentropy'</span><span class="p" style="box-sizing: border-box;">,</span>
<span class="n" style="box-sizing: border-box;">metrics</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="p" style="box-sizing: border-box;">[</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'accuracy'</span><span class="p" style="box-sizing: border-box;">])</span>
<span class="n" style="box-sizing: border-box;">Also</span><span class="p" style="box-sizing: border-box;">,</span> <span class="n" style="box-sizing: border-box;">we</span> <span class="n" style="box-sizing: border-box;">will</span> <span class="n" style="box-sizing: border-box;">run</span> <span class="n" style="box-sizing: border-box;">the</span> <span class="n" style="box-sizing: border-box;">job</span> <span class="k" style="box-sizing: border-box; color: green; font-weight: bold;">for</span> <span class="mi" style="box-sizing: border-box; color: #666666;">15</span> <span class="n" style="box-sizing: border-box;">Epochs</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span></pre>
</div>
<div>
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;"><br /></span></div>
<div>
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;"><br /></span></div>
<div>
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;"><br /></span></div>
<div>
<pre style="background-color: #f7f7f7; border-radius: 2px; border: none; box-sizing: border-box; color: #333333; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 0px; white-space: pre-wrap; word-break: break-all;"><span class="kn" style="box-sizing: border-box; color: green; font-weight: bold;">import</span> <span class="nn" style="box-sizing: border-box; color: blue; font-weight: bold;">tensorflow</span> <span class="k" style="box-sizing: border-box; color: green; font-weight: bold;">as</span> <span class="nn" style="box-sizing: border-box; color: blue; font-weight: bold;">tf</span>
<span class="kn" style="box-sizing: border-box; color: green; font-weight: bold;">import</span> <span class="nn" style="box-sizing: border-box; color: blue; font-weight: bold;">argparse</span>
<span class="kn" style="box-sizing: border-box; color: green; font-weight: bold;">import</span> <span class="nn" style="box-sizing: border-box; color: blue; font-weight: bold;">os</span>
<span class="kn" style="box-sizing: border-box; color: green; font-weight: bold;">import</span> <span class="nn" style="box-sizing: border-box; color: blue; font-weight: bold;">numpy</span> <span class="k" style="box-sizing: border-box; color: green; font-weight: bold;">as</span> <span class="nn" style="box-sizing: border-box; color: blue; font-weight: bold;">np</span>
<span class="kn" style="box-sizing: border-box; color: green; font-weight: bold;">import</span> <span class="nn" style="box-sizing: border-box; color: blue; font-weight: bold;">json</span>
<span class="kn" style="box-sizing: border-box; color: green; font-weight: bold;">from</span> <span class="nn" style="box-sizing: border-box; color: blue; font-weight: bold;">tensorflow.keras.preprocessing.image</span> <span class="kn" style="box-sizing: border-box; color: green; font-weight: bold;">import</span> <span class="n" style="box-sizing: border-box;">ImageDataGenerator</span>
<span class="kn" style="box-sizing: border-box; color: green; font-weight: bold;">from</span> <span class="nn" style="box-sizing: border-box; color: blue; font-weight: bold;">tensorflow.keras.optimizers</span> <span class="kn" style="box-sizing: border-box; color: green; font-weight: bold;">import</span> <span class="n" style="box-sizing: border-box;">RMSprop</span>
<span class="k" style="box-sizing: border-box; color: green; font-weight: bold;">def</span> <span class="nf" style="box-sizing: border-box; color: blue;">model</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">train_generator</span><span class="p" style="box-sizing: border-box;">,</span> <span class="n" style="box-sizing: border-box;">validation_generator</span><span class="p" style="box-sizing: border-box;">):</span>
<span class="sd" style="box-sizing: border-box; color: #ba2121; font-style: italic;">"""Generate a simple model"""</span>
<span class="n" style="box-sizing: border-box;">model</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">tf</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">keras</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">models</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">Sequential</span><span class="p" style="box-sizing: border-box;">([</span>
<span class="c1" style="box-sizing: border-box; color: #408080; font-style: italic;"># Note the input shape is the desired size of the image 150x150 with 3 bytes color</span>
<span class="n" style="box-sizing: border-box;">tf</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">keras</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">layers</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">Conv2D</span><span class="p" style="box-sizing: border-box;">(</span><span class="mi" style="box-sizing: border-box; color: #666666;">16</span><span class="p" style="box-sizing: border-box;">,</span> <span class="p" style="box-sizing: border-box;">(</span><span class="mi" style="box-sizing: border-box; color: #666666;">3</span><span class="p" style="box-sizing: border-box;">,</span><span class="mi" style="box-sizing: border-box; color: #666666;">3</span><span class="p" style="box-sizing: border-box;">),</span> <span class="n" style="box-sizing: border-box;">activation</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'relu'</span><span class="p" style="box-sizing: border-box;">,</span> <span class="n" style="box-sizing: border-box;">input_shape</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="p" style="box-sizing: border-box;">(</span><span class="mi" style="box-sizing: border-box; color: #666666;">150</span><span class="p" style="box-sizing: border-box;">,</span> <span class="mi" style="box-sizing: border-box; color: #666666;">150</span><span class="p" style="box-sizing: border-box;">,</span> <span class="mi" style="box-sizing: border-box; color: #666666;">3</span><span class="p" style="box-sizing: border-box;">)),</span>
<span class="n" style="box-sizing: border-box;">tf</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">keras</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">layers</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">MaxPooling2D</span><span class="p" style="box-sizing: border-box;">(</span><span class="mi" style="box-sizing: border-box; color: #666666;">2</span><span class="p" style="box-sizing: border-box;">,</span><span class="mi" style="box-sizing: border-box; color: #666666;">2</span><span class="p" style="box-sizing: border-box;">),</span>
<span class="n" style="box-sizing: border-box;">tf</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">keras</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">layers</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">Conv2D</span><span class="p" style="box-sizing: border-box;">(</span><span class="mi" style="box-sizing: border-box; color: #666666;">32</span><span class="p" style="box-sizing: border-box;">,</span> <span class="p" style="box-sizing: border-box;">(</span><span class="mi" style="box-sizing: border-box; color: #666666;">3</span><span class="p" style="box-sizing: border-box;">,</span><span class="mi" style="box-sizing: border-box; color: #666666;">3</span><span class="p" style="box-sizing: border-box;">),</span> <span class="n" style="box-sizing: border-box;">activation</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'relu'</span><span class="p" style="box-sizing: border-box;">),</span>
<span class="n" style="box-sizing: border-box;">tf</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">keras</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">layers</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">MaxPooling2D</span><span class="p" style="box-sizing: border-box;">(</span><span class="mi" style="box-sizing: border-box; color: #666666;">2</span><span class="p" style="box-sizing: border-box;">,</span><span class="mi" style="box-sizing: border-box; color: #666666;">2</span><span class="p" style="box-sizing: border-box;">),</span>
<span class="n" style="box-sizing: border-box;">tf</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">keras</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">layers</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">Conv2D</span><span class="p" style="box-sizing: border-box;">(</span><span class="mi" style="box-sizing: border-box; color: #666666;">64</span><span class="p" style="box-sizing: border-box;">,</span> <span class="p" style="box-sizing: border-box;">(</span><span class="mi" style="box-sizing: border-box; color: #666666;">3</span><span class="p" style="box-sizing: border-box;">,</span><span class="mi" style="box-sizing: border-box; color: #666666;">3</span><span class="p" style="box-sizing: border-box;">),</span> <span class="n" style="box-sizing: border-box;">activation</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'relu'</span><span class="p" style="box-sizing: border-box;">),</span>
<span class="n" style="box-sizing: border-box;">tf</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">keras</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">layers</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">MaxPooling2D</span><span class="p" style="box-sizing: border-box;">(</span><span class="mi" style="box-sizing: border-box; color: #666666;">2</span><span class="p" style="box-sizing: border-box;">,</span><span class="mi" style="box-sizing: border-box; color: #666666;">2</span><span class="p" style="box-sizing: border-box;">),</span>
<span class="c1" style="box-sizing: border-box; color: #408080; font-style: italic;"># Flatten the results to feed into a DNN</span>
<span class="n" style="box-sizing: border-box;">tf</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">keras</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">layers</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">Flatten</span><span class="p" style="box-sizing: border-box;">(),</span>
<span class="c1" style="box-sizing: border-box; color: #408080; font-style: italic;"># 512 neuron hidden layer</span>
<span class="n" style="box-sizing: border-box;">tf</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">keras</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">layers</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">Dense</span><span class="p" style="box-sizing: border-box;">(</span><span class="mi" style="box-sizing: border-box; color: #666666;">512</span><span class="p" style="box-sizing: border-box;">,</span> <span class="n" style="box-sizing: border-box;">activation</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'relu'</span><span class="p" style="box-sizing: border-box;">),</span>
<span class="c1" style="box-sizing: border-box; color: #408080; font-style: italic;"># Only 1 output neuron. It will contain a value from 0-1 where 0 for 1 class ('cats') and 1 for the other ('dogs')</span>
<span class="n" style="box-sizing: border-box;">tf</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">keras</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">layers</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">Dense</span><span class="p" style="box-sizing: border-box;">(</span><span class="mi" style="box-sizing: border-box; color: #666666;">1</span><span class="p" style="box-sizing: border-box;">,</span> <span class="n" style="box-sizing: border-box;">activation</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'sigmoid'</span><span class="p" style="box-sizing: border-box;">)</span>
<span class="p" style="box-sizing: border-box;">])</span>
<span class="n" style="box-sizing: border-box;">model</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">summary</span><span class="p" style="box-sizing: border-box;">()</span>
<span class="n" style="box-sizing: border-box;">model</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">compile</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">optimizer</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="n" style="box-sizing: border-box;">RMSprop</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">lr</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="mf" style="box-sizing: border-box; color: #666666;">0.001</span><span class="p" style="box-sizing: border-box;">),</span>
<span class="n" style="box-sizing: border-box;">loss</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'binary_crossentropy'</span><span class="p" style="box-sizing: border-box;">,</span>
<span class="n" style="box-sizing: border-box;">metrics</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="p" style="box-sizing: border-box;">[</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'accuracy'</span><span class="p" style="box-sizing: border-box;">])</span>
<span class="n" style="box-sizing: border-box;">model</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">fit</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">train_generator</span><span class="p" style="box-sizing: border-box;">,</span>
<span class="n" style="box-sizing: border-box;">validation_data</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="n" style="box-sizing: border-box;">validation_generator</span><span class="p" style="box-sizing: border-box;">,</span>
<span class="n" style="box-sizing: border-box;">steps_per_epoch</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="mi" style="box-sizing: border-box; color: #666666;">100</span><span class="p" style="box-sizing: border-box;">,</span>
<span class="n" style="box-sizing: border-box;">epochs</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="mi" style="box-sizing: border-box; color: #666666;">15</span><span class="p" style="box-sizing: border-box;">,</span>
<span class="n" style="box-sizing: border-box;">validation_steps</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="mi" style="box-sizing: border-box; color: #666666;">50</span><span class="p" style="box-sizing: border-box;">,</span>
<span class="n" style="box-sizing: border-box;">verbose</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="mi" style="box-sizing: border-box; color: #666666;">2</span><span class="p" style="box-sizing: border-box;">)</span>
<span class="k" style="box-sizing: border-box; color: green; font-weight: bold;">return</span> <span class="n" style="box-sizing: border-box;">model</span></pre>
</div>
<div>
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;"><br /></span></div>
<div>
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;"><br /></span></div>
<div>
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;"><br /></span></div>
<div>
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;">In this function, we use the Keras Image generator to load the image data. You can refer the documentation for more details. </span><a href="https://keras.io/api/preprocessing/image/" style="background-color: white; box-sizing: border-box; color: #337ab7; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px;">https://keras.io/api/preprocessing/image/</a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<pre style="background-color: #f7f7f7; border-radius: 2px; border: none; box-sizing: border-box; color: #333333; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 0px; white-space: pre-wrap; word-break: break-all;"><span class="k" style="box-sizing: border-box; color: green; font-weight: bold;">def</span> <span class="nf" style="box-sizing: border-box; color: blue;">_load_trainingandvalidation_data</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">base_dir</span><span class="p" style="box-sizing: border-box;">):</span>
<span class="sd" style="box-sizing: border-box; color: #ba2121; font-style: italic;">"""Load MNIST training data"""</span>
<span class="n" style="box-sizing: border-box;">train_dir</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">os</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">path</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">join</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">base_dir</span><span class="p" style="box-sizing: border-box;">,</span> <span class="s1" style="box-sizing: border-box; color: #ba2121;">'train'</span><span class="p" style="box-sizing: border-box;">)</span>
<span class="n" style="box-sizing: border-box;">validation_dir</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">os</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">path</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">join</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">base_dir</span><span class="p" style="box-sizing: border-box;">,</span> <span class="s1" style="box-sizing: border-box; color: #ba2121;">'validation'</span><span class="p" style="box-sizing: border-box;">)</span>
<span class="c1" style="box-sizing: border-box; color: #408080; font-style: italic;"># Directory with our training cat/dog pictures</span>
<span class="n" style="box-sizing: border-box;">train_cats_dir</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">os</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">path</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">join</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">train_dir</span><span class="p" style="box-sizing: border-box;">,</span> <span class="s1" style="box-sizing: border-box; color: #ba2121;">'cats'</span><span class="p" style="box-sizing: border-box;">)</span>
<span class="n" style="box-sizing: border-box;">train_dogs_dir</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">os</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">path</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">join</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">train_dir</span><span class="p" style="box-sizing: border-box;">,</span> <span class="s1" style="box-sizing: border-box; color: #ba2121;">'dogs'</span><span class="p" style="box-sizing: border-box;">)</span>
<span class="c1" style="box-sizing: border-box; color: #408080; font-style: italic;"># Directory with our validation cat/dog pictures</span>
<span class="n" style="box-sizing: border-box;">validation_cats_dir</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">os</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">path</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">join</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">validation_dir</span><span class="p" style="box-sizing: border-box;">,</span> <span class="s1" style="box-sizing: border-box; color: #ba2121;">'cats'</span><span class="p" style="box-sizing: border-box;">)</span>
<span class="n" style="box-sizing: border-box;">validation_dogs_dir</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">os</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">path</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">join</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">validation_dir</span><span class="p" style="box-sizing: border-box;">,</span> <span class="s1" style="box-sizing: border-box; color: #ba2121;">'dogs'</span><span class="p" style="box-sizing: border-box;">)</span>
<span class="nb" style="box-sizing: border-box; color: green;">print</span><span class="p" style="box-sizing: border-box;">(</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'total training cat images :'</span><span class="p" style="box-sizing: border-box;">,</span> <span class="nb" style="box-sizing: border-box; color: green;">len</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">os</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">listdir</span><span class="p" style="box-sizing: border-box;">(</span> <span class="n" style="box-sizing: border-box;">train_cats_dir</span> <span class="p" style="box-sizing: border-box;">)</span> <span class="p" style="box-sizing: border-box;">))</span>
<span class="nb" style="box-sizing: border-box; color: green;">print</span><span class="p" style="box-sizing: border-box;">(</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'total training dog images :'</span><span class="p" style="box-sizing: border-box;">,</span> <span class="nb" style="box-sizing: border-box; color: green;">len</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">os</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">listdir</span><span class="p" style="box-sizing: border-box;">(</span> <span class="n" style="box-sizing: border-box;">train_dogs_dir</span> <span class="p" style="box-sizing: border-box;">)</span> <span class="p" style="box-sizing: border-box;">))</span>
<span class="nb" style="box-sizing: border-box; color: green;">print</span><span class="p" style="box-sizing: border-box;">(</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'total validation cat images :'</span><span class="p" style="box-sizing: border-box;">,</span> <span class="nb" style="box-sizing: border-box; color: green;">len</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">os</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">listdir</span><span class="p" style="box-sizing: border-box;">(</span> <span class="n" style="box-sizing: border-box;">validation_cats_dir</span> <span class="p" style="box-sizing: border-box;">)</span> <span class="p" style="box-sizing: border-box;">))</span>
<span class="nb" style="box-sizing: border-box; color: green;">print</span><span class="p" style="box-sizing: border-box;">(</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'total validation dog images :'</span><span class="p" style="box-sizing: border-box;">,</span> <span class="nb" style="box-sizing: border-box; color: green;">len</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">os</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">listdir</span><span class="p" style="box-sizing: border-box;">(</span> <span class="n" style="box-sizing: border-box;">validation_dogs_dir</span> <span class="p" style="box-sizing: border-box;">)</span> <span class="p" style="box-sizing: border-box;">))</span>
<span class="c1" style="box-sizing: border-box; color: #408080; font-style: italic;"># All images will be rescaled by 1./255.</span>
<span class="n" style="box-sizing: border-box;">train_datagen</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">ImageDataGenerator</span><span class="p" style="box-sizing: border-box;">(</span> <span class="n" style="box-sizing: border-box;">rescale</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="mf" style="box-sizing: border-box; color: #666666;">1.0</span><span class="o" style="box-sizing: border-box; color: #666666;">/</span><span class="mf" style="box-sizing: border-box; color: #666666;">255.</span> <span class="p" style="box-sizing: border-box;">)</span>
<span class="n" style="box-sizing: border-box;">test_datagen</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">ImageDataGenerator</span><span class="p" style="box-sizing: border-box;">(</span> <span class="n" style="box-sizing: border-box;">rescale</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="mf" style="box-sizing: border-box; color: #666666;">1.0</span><span class="o" style="box-sizing: border-box; color: #666666;">/</span><span class="mf" style="box-sizing: border-box; color: #666666;">255.</span> <span class="p" style="box-sizing: border-box;">)</span>
<span class="c1" style="box-sizing: border-box; color: #408080; font-style: italic;"># --------------------</span>
<span class="c1" style="box-sizing: border-box; color: #408080; font-style: italic;"># Flow training images in batches of 20 using train_datagen generator</span>
<span class="c1" style="box-sizing: border-box; color: #408080; font-style: italic;"># --------------------</span>
<span class="n" style="box-sizing: border-box;">train_generator</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">train_datagen</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">flow_from_directory</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">train_dir</span><span class="p" style="box-sizing: border-box;">,</span>
<span class="n" style="box-sizing: border-box;">batch_size</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="mi" style="box-sizing: border-box; color: #666666;">20</span><span class="p" style="box-sizing: border-box;">,</span>
<span class="n" style="box-sizing: border-box;">class_mode</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'binary'</span><span class="p" style="box-sizing: border-box;">,</span>
<span class="n" style="box-sizing: border-box;">target_size</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="p" style="box-sizing: border-box;">(</span><span class="mi" style="box-sizing: border-box; color: #666666;">150</span><span class="p" style="box-sizing: border-box;">,</span> <span class="mi" style="box-sizing: border-box; color: #666666;">150</span><span class="p" style="box-sizing: border-box;">))</span>
<span class="c1" style="box-sizing: border-box; color: #408080; font-style: italic;"># --------------------</span>
<span class="c1" style="box-sizing: border-box; color: #408080; font-style: italic;"># Flow validation images in batches of 20 using test_datagen generator</span>
<span class="c1" style="box-sizing: border-box; color: #408080; font-style: italic;"># --------------------</span>
<span class="n" style="box-sizing: border-box;">validation_generator</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">test_datagen</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">flow_from_directory</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">validation_dir</span><span class="p" style="box-sizing: border-box;">,</span>
<span class="n" style="box-sizing: border-box;">batch_size</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="mi" style="box-sizing: border-box; color: #666666;">20</span><span class="p" style="box-sizing: border-box;">,</span>
<span class="n" style="box-sizing: border-box;">class_mode</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="s1" style="box-sizing: border-box; color: #ba2121;">'binary'</span><span class="p" style="box-sizing: border-box;">,</span>
<span class="n" style="box-sizing: border-box;">target_size</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="p" style="box-sizing: border-box;">(</span><span class="mi" style="box-sizing: border-box; color: #666666;">150</span><span class="p" style="box-sizing: border-box;">,</span> <span class="mi" style="box-sizing: border-box; color: #666666;">150</span><span class="p" style="box-sizing: border-box;">))</span>
<span class="k" style="box-sizing: border-box; color: green; font-weight: bold;">return</span> <span class="n" style="box-sizing: border-box;">train_generator</span><span class="p" style="box-sizing: border-box;">,</span> <span class="n" style="box-sizing: border-box;">validation_generator</span></pre>
</div>
<div>
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;"><br /></span></div>
<div>
<div style="background-color: white; box-sizing: border-box; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px;">
Since we are passing the script to sagemaker, we have to create the main method. The only relavent parameter is SM_CHANNEL_TRAINING, which we will pass as the input.</div>
<div style="background-color: white; box-sizing: border-box; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px; margin-top: 1em;">
SM_CHANNEL_TRAINING -> Is the base directory for data preparation. In our case it will be the Amazon S3 bucket path, which we have prepared above.</div>
<div style="background-color: white; box-sizing: border-box; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px; margin-top: 1em;">
<br /></div>
<pre style="background-color: #f7f7f7; border-radius: 2px; border: none; box-sizing: border-box; color: #333333; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 0px; white-space: pre-wrap; word-break: break-all;"><span class="k" style="box-sizing: border-box; color: green; font-weight: bold;">def</span> <span class="nf" style="box-sizing: border-box; color: blue;">_parse_args</span><span class="p" style="box-sizing: border-box;">():</span>
<span class="n" style="box-sizing: border-box;">parser</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">argparse</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">ArgumentParser</span><span class="p" style="box-sizing: border-box;">()</span>
<span class="c1" style="box-sizing: border-box; color: #408080; font-style: italic;"># Data, model, and output directories</span>
<span class="c1" style="box-sizing: border-box; color: #408080; font-style: italic;"># model_dir is always passed in from SageMaker. By default this is a S3 path under the default bucket.</span>
<span class="n" style="box-sizing: border-box;">parser</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">add_argument</span><span class="p" style="box-sizing: border-box;">(</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'--model_dir'</span><span class="p" style="box-sizing: border-box;">,</span> <span class="nb" style="box-sizing: border-box; color: green;">type</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="nb" style="box-sizing: border-box; color: green;">str</span><span class="p" style="box-sizing: border-box;">)</span>
<span class="n" style="box-sizing: border-box;">parser</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">add_argument</span><span class="p" style="box-sizing: border-box;">(</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'--sm-model-dir'</span><span class="p" style="box-sizing: border-box;">,</span> <span class="nb" style="box-sizing: border-box; color: green;">type</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="nb" style="box-sizing: border-box; color: green;">str</span><span class="p" style="box-sizing: border-box;">,</span> <span class="n" style="box-sizing: border-box;">default</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="n" style="box-sizing: border-box;">os</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">environ</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">get</span><span class="p" style="box-sizing: border-box;">(</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'SM_MODEL_DIR'</span><span class="p" style="box-sizing: border-box;">))</span>
<span class="n" style="box-sizing: border-box;">parser</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">add_argument</span><span class="p" style="box-sizing: border-box;">(</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'--train'</span><span class="p" style="box-sizing: border-box;">,</span> <span class="nb" style="box-sizing: border-box; color: green;">type</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="nb" style="box-sizing: border-box; color: green;">str</span><span class="p" style="box-sizing: border-box;">,</span> <span class="n" style="box-sizing: border-box;">default</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="n" style="box-sizing: border-box;">os</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">environ</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">get</span><span class="p" style="box-sizing: border-box;">(</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'SM_CHANNEL_TRAINING'</span><span class="p" style="box-sizing: border-box;">))</span>
<span class="n" style="box-sizing: border-box;">parser</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">add_argument</span><span class="p" style="box-sizing: border-box;">(</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'--hosts'</span><span class="p" style="box-sizing: border-box;">,</span> <span class="nb" style="box-sizing: border-box; color: green;">type</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="nb" style="box-sizing: border-box; color: green;">list</span><span class="p" style="box-sizing: border-box;">,</span> <span class="n" style="box-sizing: border-box;">default</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="n" style="box-sizing: border-box;">json</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">loads</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">os</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">environ</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">get</span><span class="p" style="box-sizing: border-box;">(</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'SM_HOSTS'</span><span class="p" style="box-sizing: border-box;">)))</span>
<span class="n" style="box-sizing: border-box;">parser</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">add_argument</span><span class="p" style="box-sizing: border-box;">(</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'--current-host'</span><span class="p" style="box-sizing: border-box;">,</span> <span class="nb" style="box-sizing: border-box; color: green;">type</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="nb" style="box-sizing: border-box; color: green;">str</span><span class="p" style="box-sizing: border-box;">,</span> <span class="n" style="box-sizing: border-box;">default</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="n" style="box-sizing: border-box;">os</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">environ</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">get</span><span class="p" style="box-sizing: border-box;">(</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'SM_CURRENT_HOST'</span><span class="p" style="box-sizing: border-box;">))</span>
<span class="k" style="box-sizing: border-box; color: green; font-weight: bold;">return</span> <span class="n" style="box-sizing: border-box;">parser</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">parse_known_args</span><span class="p" style="box-sizing: border-box;">()</span>
<span class="k" style="box-sizing: border-box; color: green; font-weight: bold;">if</span> <span class="vm" style="box-sizing: border-box; color: #19177c;">__name__</span> <span class="o" style="box-sizing: border-box; color: #666666;">==</span> <span class="s2" style="box-sizing: border-box; color: #ba2121;">"__main__"</span><span class="p" style="box-sizing: border-box;">:</span>
<span class="n" style="box-sizing: border-box;">args</span><span class="p" style="box-sizing: border-box;">,</span> <span class="n" style="box-sizing: border-box;">unknown</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">_parse_args</span><span class="p" style="box-sizing: border-box;">()</span>
<span class="n" style="box-sizing: border-box;">train_generator</span><span class="p" style="box-sizing: border-box;">,</span> <span class="n" style="box-sizing: border-box;">validation_generator</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">_load_trainingandvalidation_data</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">args</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">train</span><span class="p" style="box-sizing: border-box;">)</span>
<span class="c1" style="box-sizing: border-box; color: #408080; font-style: italic;">#eval_data, eval_labels = _load_testing_data(args.train)</span>
<span class="n" style="box-sizing: border-box;">mnist_classifier</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">model</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">train_generator</span><span class="p" style="box-sizing: border-box;">,</span><span class="n" style="box-sizing: border-box;">validation_generator</span><span class="p" style="box-sizing: border-box;">)</span>
<span class="k" style="box-sizing: border-box; color: green; font-weight: bold;">if</span> <span class="n" style="box-sizing: border-box;">args</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">current_host</span> <span class="o" style="box-sizing: border-box; color: #666666;">==</span> <span class="n" style="box-sizing: border-box;">args</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">hosts</span><span class="p" style="box-sizing: border-box;">[</span><span class="mi" style="box-sizing: border-box; color: #666666;">0</span><span class="p" style="box-sizing: border-box;">]:</span>
<span class="c1" style="box-sizing: border-box; color: #408080; font-style: italic;"># save model to an S3 directory with version number '00000001'</span>
<span class="n" style="box-sizing: border-box;">mnist_classifier</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">save</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">os</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">path</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">join</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">args</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">sm_model_dir</span><span class="p" style="box-sizing: border-box;">,</span> <span class="s1" style="box-sizing: border-box; color: #ba2121;">'000000001'</span><span class="p" style="box-sizing: border-box;">),</span> <span class="s1" style="box-sizing: border-box; color: #ba2121;">'my_model.h5'</span><span class="p" style="box-sizing: border-box;">)</span></pre>
</div>
<div>
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;"><br /></span></div>
<div>
You can copy the entire script. catdog.py</div>
<div>
<br /></div>
<div>
<u><b>Part3:</b></u></div>
<div>
<div style="background-color: white; box-sizing: border-box; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px;">
<br /></div>
<div style="background-color: white; box-sizing: border-box; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px;">
Next we will create the Amazon Sagemaker Launcher script. This script will using catdog.py to submit the training job.</div>
<ol style="background-color: white; box-sizing: border-box; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px; list-style-image: initial; list-style-position: initial; margin-bottom: 9px; margin-top: 1em; padding-left: 2em;">
<li style="box-sizing: border-box;">Submit the training job</li>
<li style="box-sizing: border-box;">Deploy the Tensorflow model</li>
<li style="box-sizing: border-box;">Test the model</li>
</ol>
<div>
<span style="font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif;"><span style="font-size: 14px;"><br /></span></span></div>
</div>
<div>
<pre style="background-color: #f7f7f7; border-radius: 2px; border: none; box-sizing: border-box; color: #333333; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 0px; white-space: pre-wrap; word-break: break-all;"><span class="kn" style="box-sizing: border-box; color: green; font-weight: bold;">import</span> <span class="nn" style="box-sizing: border-box; color: blue; font-weight: bold;">os</span>
<span class="kn" style="box-sizing: border-box; color: green; font-weight: bold;">import</span> <span class="nn" style="box-sizing: border-box; color: blue; font-weight: bold;">sagemaker</span>
<span class="kn" style="box-sizing: border-box; color: green; font-weight: bold;">from</span> <span class="nn" style="box-sizing: border-box; color: blue; font-weight: bold;">sagemaker</span> <span class="kn" style="box-sizing: border-box; color: green; font-weight: bold;">import</span> <span class="n" style="box-sizing: border-box;">get_execution_role</span>
<span class="n" style="box-sizing: border-box;">sagemaker_session</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">sagemaker</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">Session</span><span class="p" style="box-sizing: border-box;">()</span>
<span class="n" style="box-sizing: border-box;">role</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">get_execution_role</span><span class="p" style="box-sizing: border-box;">()</span>
<span class="n" style="box-sizing: border-box;">region</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">sagemaker_session</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">boto_session</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">region_name</span></pre>
</div>
<div>
<br /></div>
<div>
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;">Set the AWS S3 Bucket name</span></div>
<div>
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;"><br /></span></div>
<div>
<pre style="background-color: #f7f7f7; border-radius: 2px; border: none; box-sizing: border-box; color: #333333; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 0px; white-space: pre-wrap; word-break: break-all;"><span class="n" style="box-sizing: border-box;">training_data_uri</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="s1" style="box-sizing: border-box; color: #ba2121;">'s3://sagemaker-05may2020842'</span></pre>
</div>
<div>
<br /></div>
<div>
<pre style="background-color: #f7f7f7; border-radius: 2px; border: none; box-sizing: border-box; color: #333333; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 0px; white-space: pre-wrap; word-break: break-all;"><span class="c1" style="box-sizing: border-box; color: #408080; font-style: italic;"># TensorFlow 2.1 script </span>
<span class="o" style="box-sizing: border-box; color: #666666;">!</span>pygmentize <span class="s1" style="box-sizing: border-box; color: #ba2121;">'catdog.py'</span></pre>
<pre style="background-color: #f7f7f7; border-radius: 2px; border: none; box-sizing: border-box; color: #333333; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 0px; white-space: pre-wrap; word-break: break-all;"><span class="s1" style="box-sizing: border-box; color: #ba2121;">
</span></pre>
<pre style="background-color: #f7f7f7; border-radius: 2px; border: none; box-sizing: border-box; color: #333333; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 0px; white-space: pre-wrap; word-break: break-all;"><span class="s1" style="box-sizing: border-box; color: #ba2121;"><pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; color: black; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 1px 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all;"><span style="font-size: x-small;"><span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">import</span> <span class="ansi-cyan-fg ansi-underline" style="box-sizing: border-box; color: #60c6c8;">tensorflow</span> <span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">as</span> <span class="ansi-cyan-fg ansi-underline" style="box-sizing: border-box; color: #60c6c8;">tf</span>
<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">import</span> <span class="ansi-cyan-fg ansi-underline" style="box-sizing: border-box; color: #60c6c8;">argparse</span>
<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">import</span> <span class="ansi-cyan-fg ansi-underline" style="box-sizing: border-box; color: #60c6c8;">os</span>
<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">import</span> <span class="ansi-cyan-fg ansi-underline" style="box-sizing: border-box; color: #60c6c8;">numpy</span> <span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">as</span> <span class="ansi-cyan-fg ansi-underline" style="box-sizing: border-box; color: #60c6c8;">np</span>
<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">import</span> <span class="ansi-cyan-fg ansi-underline" style="box-sizing: border-box; color: #60c6c8;">json</span>
<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">from</span> <span class="ansi-cyan-fg ansi-underline" style="box-sizing: border-box; color: #60c6c8;">tensorflow</span><span class="ansi-cyan-fg ansi-underline" style="box-sizing: border-box; color: #60c6c8;">.</span><span class="ansi-cyan-fg ansi-underline" style="box-sizing: border-box; color: #60c6c8;">keras</span><span class="ansi-cyan-fg ansi-underline" style="box-sizing: border-box; color: #60c6c8;">.</span><span class="ansi-cyan-fg ansi-underline" style="box-sizing: border-box; color: #60c6c8;">preprocessing</span><span class="ansi-cyan-fg ansi-underline" style="box-sizing: border-box; color: #60c6c8;">.</span><span class="ansi-cyan-fg ansi-underline" style="box-sizing: border-box; color: #60c6c8;">image</span> <span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">import</span> ImageDataGenerator
<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">from</span> <span class="ansi-cyan-fg ansi-underline" style="box-sizing: border-box; color: #60c6c8;">tensorflow</span><span class="ansi-cyan-fg ansi-underline" style="box-sizing: border-box; color: #60c6c8;">.</span><span class="ansi-cyan-fg ansi-underline" style="box-sizing: border-box; color: #60c6c8;">keras</span><span class="ansi-cyan-fg ansi-underline" style="box-sizing: border-box; color: #60c6c8;">.</span><span class="ansi-cyan-fg ansi-underline" style="box-sizing: border-box; color: #60c6c8;">optimizers</span> <span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">import</span> RMSprop
<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">def</span> <span class="ansi-green-fg" style="box-sizing: border-box; color: #00a250;">model</span>(train_generator, validation_generator):
<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">"""Generate a simple model"""</span>
model = tf.keras.models.Sequential([
<span class="ansi-white-fg" style="box-sizing: border-box; color: #c5c1b4;"># Note the input shape is the desired size of the image 150x150 with 3 bytes color</span>
tf.keras.layers.Conv2D(<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">16</span>, (<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">3</span>,<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">3</span>), activation=<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">relu</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>, input_shape=(<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">150</span>, <span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">150</span>, <span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">3</span>)),
tf.keras.layers.MaxPooling2D(<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">2</span>,<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">2</span>),
tf.keras.layers.Conv2D(<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">32</span>, (<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">3</span>,<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">3</span>), activation=<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">relu</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>),
tf.keras.layers.MaxPooling2D(<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">2</span>,<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">2</span>),
tf.keras.layers.Conv2D(<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">64</span>, (<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">3</span>,<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">3</span>), activation=<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">relu</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>),
tf.keras.layers.MaxPooling2D(<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">2</span>,<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">2</span>),
<span class="ansi-white-fg" style="box-sizing: border-box; color: #c5c1b4;"># Flatten the results to feed into a DNN</span>
tf.keras.layers.Flatten(),
<span class="ansi-white-fg" style="box-sizing: border-box; color: #c5c1b4;"># 512 neuron hidden layer</span>
tf.keras.layers.Dense(<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">512</span>, activation=<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">relu</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>),
<span class="ansi-white-fg" style="box-sizing: border-box; color: #c5c1b4;"># Only 1 output neuron. It will contain a value from 0-1 where 0 for 1 class ('cats') and 1 for the other ('dogs')</span>
tf.keras.layers.Dense(<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">1</span>, activation=<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">sigmoid</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>)
])
model.summary()
model.compile(optimizer=RMSprop(lr=<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">0.001</span>),
loss=<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">binary_crossentropy</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>,
metrics = [<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">accuracy</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>])
model.fit(train_generator,
validation_data=validation_generator,
steps_per_epoch=<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">100</span>,
epochs=<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">15</span>,
validation_steps=<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">50</span>,
verbose=<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">2</span>)
<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">return</span> model
<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">def</span> <span class="ansi-green-fg" style="box-sizing: border-box; color: #00a250;">_load_trainingandvalidation_data</span>(base_dir):
<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">"""Load MNIST training data"""</span>
train_dir = os.path.join(base_dir, <span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">train</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>)
validation_dir = os.path.join(base_dir, <span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">validation</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>)
<span class="ansi-white-fg" style="box-sizing: border-box; color: #c5c1b4;"># Directory with our training cat/dog pictures</span>
train_cats_dir = os.path.join(train_dir, <span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">cats</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>)
train_dogs_dir = os.path.join(train_dir, <span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">dogs</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>)
<span class="ansi-white-fg" style="box-sizing: border-box; color: #c5c1b4;"># Directory with our validation cat/dog pictures</span>
validation_cats_dir = os.path.join(validation_dir, <span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">cats</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>)
validation_dogs_dir = os.path.join(validation_dir, <span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">dogs</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>)
<span class="ansi-cyan-fg" style="box-sizing: border-box; color: #60c6c8;">print</span>(<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">total training cat images :</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>, <span class="ansi-cyan-fg" style="box-sizing: border-box; color: #60c6c8;">len</span>(os.listdir( train_cats_dir ) ))
<span class="ansi-cyan-fg" style="box-sizing: border-box; color: #60c6c8;">print</span>(<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">total training dog images :</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>, <span class="ansi-cyan-fg" style="box-sizing: border-box; color: #60c6c8;">len</span>(os.listdir( train_dogs_dir ) ))
<span class="ansi-cyan-fg" style="box-sizing: border-box; color: #60c6c8;">print</span>(<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">total validation cat images :</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>, <span class="ansi-cyan-fg" style="box-sizing: border-box; color: #60c6c8;">len</span>(os.listdir( validation_cats_dir ) ))
<span class="ansi-cyan-fg" style="box-sizing: border-box; color: #60c6c8;">print</span>(<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">total validation dog images :</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>, <span class="ansi-cyan-fg" style="box-sizing: border-box; color: #60c6c8;">len</span>(os.listdir( validation_dogs_dir ) ))
<span class="ansi-white-fg" style="box-sizing: border-box; color: #c5c1b4;"># All images will be rescaled by 1./255.</span>
train_datagen = ImageDataGenerator( rescale = <span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">1.0</span>/<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">255.</span> )
test_datagen = ImageDataGenerator( rescale = <span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">1.0</span>/<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">255.</span> )
<span class="ansi-white-fg" style="box-sizing: border-box; color: #c5c1b4;"># --------------------</span>
<span class="ansi-white-fg" style="box-sizing: border-box; color: #c5c1b4;"># Flow training images in batches of 20 using train_datagen generator</span>
<span class="ansi-white-fg" style="box-sizing: border-box; color: #c5c1b4;"># --------------------</span>
train_generator = train_datagen.flow_from_directory(train_dir,
batch_size=<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">20</span>,
class_mode=<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">binary</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>,
target_size=(<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">150</span>, <span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">150</span>))
<span class="ansi-white-fg" style="box-sizing: border-box; color: #c5c1b4;"># --------------------</span>
<span class="ansi-white-fg" style="box-sizing: border-box; color: #c5c1b4;"># Flow validation images in batches of 20 using test_datagen generator</span>
<span class="ansi-white-fg" style="box-sizing: border-box; color: #c5c1b4;"># --------------------</span>
validation_generator = test_datagen.flow_from_directory(validation_dir,
batch_size=<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">20</span>,
class_mode = <span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">binary</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>,
target_size = (<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">150</span>, <span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">150</span>))
<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">return</span> train_generator, validation_generator
<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">def</span> <span class="ansi-green-fg" style="box-sizing: border-box; color: #00a250;">_parse_args</span>():
parser = argparse.ArgumentParser()
<span class="ansi-white-fg" style="box-sizing: border-box; color: #c5c1b4;"># Data, model, and output directories</span>
<span class="ansi-white-fg" style="box-sizing: border-box; color: #c5c1b4;"># model_dir is always passed in from SageMaker. By default this is a S3 path under the default bucket.</span>
parser.add_argument(<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">--model_dir</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>, <span class="ansi-cyan-fg" style="box-sizing: border-box; color: #60c6c8;">type</span>=<span class="ansi-cyan-fg" style="box-sizing: border-box; color: #60c6c8;">str</span>)
parser.add_argument(<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">--sm-model-dir</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>, <span class="ansi-cyan-fg" style="box-sizing: border-box; color: #60c6c8;">type</span>=<span class="ansi-cyan-fg" style="box-sizing: border-box; color: #60c6c8;">str</span>, default=os.environ.get(<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">SM_MODEL_DIR</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>))
parser.add_argument(<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">--train</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>, <span class="ansi-cyan-fg" style="box-sizing: border-box; color: #60c6c8;">type</span>=<span class="ansi-cyan-fg" style="box-sizing: border-box; color: #60c6c8;">str</span>, default=os.environ.get(<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">SM_CHANNEL_TRAINING</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>))
parser.add_argument(<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">--hosts</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>, <span class="ansi-cyan-fg" style="box-sizing: border-box; color: #60c6c8;">type</span>=<span class="ansi-cyan-fg" style="box-sizing: border-box; color: #60c6c8;">list</span>, default=json.loads(os.environ.get(<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">SM_HOSTS</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>)))
parser.add_argument(<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">--current-host</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>, <span class="ansi-cyan-fg" style="box-sizing: border-box; color: #60c6c8;">type</span>=<span class="ansi-cyan-fg" style="box-sizing: border-box; color: #60c6c8;">str</span>, default=os.environ.get(<span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">SM_CURRENT_HOST</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>))
<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">return</span> parser.parse_known_args()
<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">if</span> <span class="ansi-red-fg" style="box-sizing: border-box; color: #e75c58;">__name__</span> == <span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">"</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">__main__</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">"</span>:
args, unknown = _parse_args()
train_generator, validation_generator = _load_trainingandvalidation_data(args.train)
<span class="ansi-white-fg" style="box-sizing: border-box; color: #c5c1b4;">#eval_data, eval_labels = _load_testing_data(args.train)</span>
mnist_classifier = model(train_generator,validation_generator)
<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">if</span> args.current_host == args.hosts[<span class="ansi-blue-fg" style="box-sizing: border-box; color: #208ffb;">0</span>]:
<span class="ansi-white-fg" style="box-sizing: border-box; color: #c5c1b4;"># save model to an S3 directory with version number '00000001'</span>
mnist_classifier.save(os.path.join(args.sm_model_dir, <span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">000000001</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>), <span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">my_model.h5</span><span class="ansi-yellow-fg" style="box-sizing: border-box; color: #ddb62b;">'</span>)</span></pre>
</span></pre>
</div>
<div>
<br /></div>
<div>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 1px 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all;"><span style="font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; white-space: normal;">Create the Tensorflow estimator using Amazon SDK. Note the entry_point paramater is the path of the script file which we created above.</span></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 1px 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all;"><span style="font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; white-space: normal;">
</span></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 1px 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all;"><pre style="background-color: #f7f7f7; border-radius: 2px; border: none; box-sizing: border-box; color: #333333; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 0px; white-space: pre-wrap; word-break: break-all;"><span class="kn" style="box-sizing: border-box; color: green; font-weight: bold;">from</span> <span class="nn" style="box-sizing: border-box; color: blue; font-weight: bold;">sagemaker.tensorflow</span> <span class="kn" style="box-sizing: border-box; color: green; font-weight: bold;">import</span> <span class="n" style="box-sizing: border-box;">TensorFlow</span>
<span class="n" style="box-sizing: border-box;">mnist_estimator2</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">TensorFlow</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">entry_point</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'catdog.py'</span><span class="p" style="box-sizing: border-box;">,</span>
<span class="n" style="box-sizing: border-box;">role</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="n" style="box-sizing: border-box;">role</span><span class="p" style="box-sizing: border-box;">,</span>
<span class="n" style="box-sizing: border-box;">train_instance_count</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="mi" style="box-sizing: border-box; color: #666666;">2</span><span class="p" style="box-sizing: border-box;">,</span>
<span class="n" style="box-sizing: border-box;">train_instance_type</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'ml.m4.xlarge'</span><span class="p" style="box-sizing: border-box;">,</span>
<span class="n" style="box-sizing: border-box;">framework_version</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'2.1.0'</span><span class="p" style="box-sizing: border-box;">,</span>
<span class="n" style="box-sizing: border-box;">py_version</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'py3'</span><span class="p" style="box-sizing: border-box;">,</span>
<span class="n" style="box-sizing: border-box;">distributions</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="p" style="box-sizing: border-box;">{</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'parameter_server'</span><span class="p" style="box-sizing: border-box;">:</span> <span class="p" style="box-sizing: border-box;">{</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'enabled'</span><span class="p" style="box-sizing: border-box;">:</span> <span class="kc" style="box-sizing: border-box; color: green; font-weight: bold;">True</span><span class="p" style="box-sizing: border-box;">}})</span></pre>
</pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 1px 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all;"><span style="font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; white-space: normal;">
</span></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 1px 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all;">Fit method will actually submit the Job to start the training on the requested instance type.</pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 1px 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 1px 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all;"><pre style="background-color: #f7f7f7; border-radius: 2px; border: none; box-sizing: border-box; color: #333333; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 0px; white-space: pre-wrap; word-break: break-all;"><span class="n" style="box-sizing: border-box;">mnist_estimator2</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">fit</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">training_data_uri</span><span class="p" style="box-sizing: border-box;">)</span></pre>
</pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 1px 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all;"><span style="font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; white-space: normal;">
</span></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 1px 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all;"><pre style="border-radius: 0px; border: 0px; box-sizing: border-box; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 1px 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all;"><i><span style="font-size: x-small;">2020-05-09 16:38:38 Uploading - Uploading generated training model
2020-05-09 16:38:38 Completed - Training job completed
Training seconds: 954
Billable seconds: 954</span></i></pre>
</pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 1px 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all;"><span style="font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; white-space: normal;">
</span></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 1px 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all;">Once the training is completed, you can simply run below command to get the model deployed.</pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 1px 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 1px 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all;"><pre style="background-color: #f7f7f7; border-radius: 2px; border: none; box-sizing: border-box; color: #333333; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 0px; white-space: pre-wrap; word-break: break-all;"><span class="n" style="box-sizing: border-box;">predictor2</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">mnist_estimator2</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">deploy</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">initial_instance_count</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="mi" style="box-sizing: border-box; color: #666666;">1</span><span class="p" style="box-sizing: border-box;">,</span> <span class="n" style="box-sizing: border-box;">instance_type</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'ml.m4.xlarge'</span><span class="p" style="box-sizing: border-box;">)</span></pre>
</pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 1px 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 1px 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all;"><div style="box-sizing: border-box; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; white-space: normal;">
Once the model is deployed. You can use the predict method to test the model in serving mode. For testing you can download the free images from <a href="https://pixabay.com/" style="background-color: transparent; box-sizing: border-box; color: #337ab7;">https://pixabay.com/</a></div>
<div style="box-sizing: border-box; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; margin-top: 1em; white-space: normal;">
Also, you will need to convert the raw image to numpy array. Please refer below code for ref.</div>
</pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 1px 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 1px 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all;"><pre style="background-color: #f7f7f7; border-radius: 2px; border: none; box-sizing: border-box; color: #333333; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 0px; white-space: pre-wrap; word-break: break-all;"><span class="kn" style="box-sizing: border-box; color: green; font-weight: bold;">from</span> <span class="nn" style="box-sizing: border-box; color: blue; font-weight: bold;">keras.preprocessing</span> <span class="kn" style="box-sizing: border-box; color: green; font-weight: bold;">import</span> <span class="n" style="box-sizing: border-box;">image</span>
<span class="n" style="box-sizing: border-box;">path</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'bulldog-1047518_640.jpg'</span>
<span class="n" style="box-sizing: border-box;">img</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="n" style="box-sizing: border-box;">image</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">load_img</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">path</span><span class="p" style="box-sizing: border-box;">,</span> <span class="n" style="box-sizing: border-box;">target_size</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="p" style="box-sizing: border-box;">(</span><span class="mi" style="box-sizing: border-box; color: #666666;">150</span><span class="p" style="box-sizing: border-box;">,</span> <span class="mi" style="box-sizing: border-box; color: #666666;">150</span><span class="p" style="box-sizing: border-box;">))</span>
<span class="n" style="box-sizing: border-box;">x</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="n" style="box-sizing: border-box;">image</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">img_to_array</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">img</span><span class="p" style="box-sizing: border-box;">)</span>
<span class="n" style="box-sizing: border-box;">x</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="n" style="box-sizing: border-box;">np</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">expand_dims</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">x</span><span class="p" style="box-sizing: border-box;">,</span> <span class="n" style="box-sizing: border-box;">axis</span><span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="mi" style="box-sizing: border-box; color: #666666;">0</span><span class="p" style="box-sizing: border-box;">)</span>
<span class="n" style="box-sizing: border-box;">images</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span> <span class="n" style="box-sizing: border-box;">np</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">vstack</span><span class="p" style="box-sizing: border-box;">([</span><span class="n" style="box-sizing: border-box;">x</span><span class="p" style="box-sizing: border-box;">])</span>
<span class="n" style="box-sizing: border-box;">predictions2</span> <span class="o" style="box-sizing: border-box; color: #666666;">=</span><span class="n" style="box-sizing: border-box;">predictor2</span><span class="o" style="box-sizing: border-box; color: #666666;">.</span><span class="n" style="box-sizing: border-box;">predict</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">images</span><span class="p" style="box-sizing: border-box;">)</span>
<span class="nb" style="box-sizing: border-box; color: green;">print</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">predictions2</span><span class="p" style="box-sizing: border-box;">[</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'predictions'</span><span class="p" style="box-sizing: border-box;">][</span><span class="mi" style="box-sizing: border-box; color: #666666;">0</span><span class="p" style="box-sizing: border-box;">])</span>
<span class="k" style="box-sizing: border-box; color: green; font-weight: bold;">if</span> <span class="n" style="box-sizing: border-box;">predictions2</span><span class="p" style="box-sizing: border-box;">[</span><span class="s1" style="box-sizing: border-box; color: #ba2121;">'predictions'</span><span class="p" style="box-sizing: border-box;">][</span><span class="mi" style="box-sizing: border-box; color: #666666;">0</span><span class="p" style="box-sizing: border-box;">][</span><span class="mi" style="box-sizing: border-box; color: #666666;">0</span><span class="p" style="box-sizing: border-box;">]</span><span class="o" style="box-sizing: border-box; color: #666666;">></span><span class="mi" style="box-sizing: border-box; color: #666666;">0</span><span class="p" style="box-sizing: border-box;">:</span>
<span class="nb" style="box-sizing: border-box; color: green;">print</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">path</span> <span class="o" style="box-sizing: border-box; color: #666666;">+</span> <span class="s2" style="box-sizing: border-box; color: #ba2121;">" is a dog"</span><span class="p" style="box-sizing: border-box;">)</span>
<span class="k" style="box-sizing: border-box; color: green; font-weight: bold;">else</span><span class="p" style="box-sizing: border-box;">:</span>
<span class="nb" style="box-sizing: border-box; color: green;">print</span><span class="p" style="box-sizing: border-box;">(</span><span class="n" style="box-sizing: border-box;">path</span> <span class="o" style="box-sizing: border-box; color: #666666;">+</span> <span class="s2" style="box-sizing: border-box; color: #ba2121;">" is a cat"</span><span class="p" style="box-sizing: border-box;">)</span></pre>
</pre>
</div>
<div>
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;"><br /></span></div>
</div>
<div>
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;"><br /></span></div>
<div>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow-wrap: break-word; overflow: auto; padding: 1px 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all;">[1.0]
bulldog-1047518_640.jpg is a dog</pre>
</div>
<div>
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;"><br /></span></div>
<div>
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;">Picture used: </span></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-Dhpp12JZgfc/XrcIMwotCzI/AAAAAAABGLg/GQhbzqy3oeM92Cb7dqJwL1UuwWpayVNxgCLcBGAsYHQ/s1600/bulldog-1047518_640.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="426" data-original-width="640" height="133" src="https://1.bp.blogspot.com/-Dhpp12JZgfc/XrcIMwotCzI/AAAAAAABGLg/GQhbzqy3oeM92Cb7dqJwL1UuwWpayVNxgCLcBGAsYHQ/s200/bulldog-1047518_640.jpg" width="200" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div>
<span style="background-color: white; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 14px;"><br /></span></div>
</div>
Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-51618800847423571512020-03-25T21:45:00.002+05:302020-03-25T21:45:55.820+05:30How to enable CORS on Axway API & use postman to verify if the CORS is enabled<div dir="ltr" style="text-align: left;" trbidi="on">
After a long break from the blog posting, I have decided to start posting again.<br />
<div>
<br /></div>
<div>
In this post we will demo how to enable CORS in Axway API Gateway and use Postman to verify whether the <b>Cross-Origin Resource Sharing (CORS)</b> are enabled on the API or not.</div>
<div>
<br /></div>
<div>
You need to enable the CORS setting on the API, if you are planning to call the API from javascript which is running in browser. For more details on CORS, please check below links</div>
<div>
<br /></div>
<div>
<a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS">https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS</a></div>
<div>
<br /></div>
<div>
<u><b>To enable CORS on Axway API</b></u></div>
<div>
<u><br /></u></div>
<div>
By Default, CORS are enabled for all domains for the frontend API in Axway. You can verify the setting on <b>Frontend API -> Inbound -> Advance tab</b></div>
<div>
<br />
<i>Enable CORS from all domains </i><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-p057g4GefNc/XnuAqX1YbGI/AAAAAAABEUE/IeM7PnrBgpIISyEQwEBNDpH8acR9VoGiQCLcBGAsYHQ/s1600/Capture.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="122" data-original-width="409" height="95" src="https://1.bp.blogspot.com/-p057g4GefNc/XnuAqX1YbGI/AAAAAAABEUE/IeM7PnrBgpIISyEQwEBNDpH8acR9VoGiQCLcBGAsYHQ/s320/Capture.PNG" width="320" /></a></div>
<br /></div>
<div>
<br />
<br />
<br />
<br />
<br />
<br />
If you want to restrict or configure specific CORS setting, you can create the CORS profile on frontend API.<br />
Click on CORS profile, Add a new CORS profile, with ORIGIN URL or ALLOWED HEADER's<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-Bds1FpmZQGI/XnuBitJ-nXI/AAAAAAABEUM/h-TedN7GN8c_aAbqMQHP6pyr-VIqvrqYQCLcBGAsYHQ/s1600/Capture.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="396" data-original-width="940" height="134" src="https://1.bp.blogspot.com/-Bds1FpmZQGI/XnuBitJ-nXI/AAAAAAABEUM/h-TedN7GN8c_aAbqMQHP6pyr-VIqvrqYQCLcBGAsYHQ/s320/Capture.PNG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
You will need to select the configured profile, please refer below screenshot.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-_KjSKRf4EwQ/XnuCJoD8sjI/AAAAAAABEUY/nGBsU7QlteguIXqJkJr3JOt8Qg7Ci4fDwCLcBGAsYHQ/s1600/Capture.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="112" data-original-width="480" height="74" src="https://1.bp.blogspot.com/-_KjSKRf4EwQ/XnuCJoD8sjI/AAAAAAABEUY/nGBsU7QlteguIXqJkJr3JOt8Qg7Ci4fDwCLcBGAsYHQ/s320/Capture.PNG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<b><u>To test or verify is the CORS is enabled:</u></b><br />
<b><u><br /></u></b>
In order to verify the CORS, browser first make the OPTIONS call to the API. Based on the headers in the response, browser will decide whether to allow the actual call or block the call.<br />
<br />
Thus, we need to make the option call from the POSTMAN to verify if we are getting the required header's in response.<br />
<br />
Make a note, that while calling OPTIONS call from POSTMAN, you will need to set <b>Origin </b>header to the hostname that is allowed on the API. Based on the configuration on the API, if the hostname is allowed, it will return the same hostname in <b>Access-Control-Allow-Origin</b> response header.<br />
<br />
If you are able to get the <b>Access-Control-Allow-Origin </b>header in response, the browser should allow the Javascript call to go through.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-vpMr_E01DGk/XnuDXYT28bI/AAAAAAABEUk/ESdamT4o1VQa3_6abLsnFfHpq8SqEsHRgCLcBGAsYHQ/s1600/Capture.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="550" data-original-width="719" height="244" src="https://1.bp.blogspot.com/-vpMr_E01DGk/XnuDXYT28bI/AAAAAAABEUk/ESdamT4o1VQa3_6abLsnFfHpq8SqEsHRgCLcBGAsYHQ/s320/Capture.PNG" width="320" /></a></div>
<br />
<br /></div>
</div>
Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-32746029394385324642017-01-01T08:55:00.003+05:302017-01-01T08:55:36.978+05:30Getting started with Oracle SOA Cloud Service: Deploying First DB Adapter Service on Cloud<div dir="ltr" style="text-align: left;" trbidi="on">
In this post we will deploy the simple database adapter service. This service acts like a get Customer service, which pulls the data from Database.<br />
<br />
Since we already configured the Oracle Database Service,<br />
<br />
1, I created a CUSTOMER TABLE in Oracle DB cloud.<br />
<br />
2. I created a Datasource in weblogic console point to the DB cloud service.<br />
<br />
3. I created the JCA connection in DBAdapter Lib under deployment and upgraded the lib.<br />
<br />
4. I Created a simple BPEL process to get the Cust ID and Fetch the Customer Data from Table.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-zjTo5ZjmozU/WGh1MRQfgbI/AAAAAAAAcdU/sQr51315UnIlIHixXCghi-qGYbTVAm89wCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="61" src="https://1.bp.blogspot.com/-zjTo5ZjmozU/WGh1MRQfgbI/AAAAAAAAcdU/sQr51315UnIlIHixXCghi-qGYbTVAm89wCLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
5. Deploy the composite through EM console, same like a on-premise SOA server.<br />
<br />
6. Once deployed, test the composite, and you will get the WSDL URL. This URL is over the public domain, so i just added the URL in SOAP UI and tested.<br />
<br />
http://oc-****.compute.oraclecloud.com:8080/soa-infra/services/MyPartition/GetCustomer/fetchcustbpel_client?wsdl<br />
<br />
Though it was a simple service I though it will take a bit longer to provide the response, but I see that the response is pretty quick, avg response time was around <b>180ms.</b><br />
<b><br /></b>
Apart from some restriction or changes, using Oracle SOA Cloud Service is almost similar to using On-Premise SOA server, but it take away all the admin/maintenance activities.<br />
<br /></div>
Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-7226941277401316652016-12-31T09:02:00.002+05:302016-12-31T09:02:25.992+05:30Getting started with Oracle SOA Cloud Service Final Part 3: Configure Oracle SOA Cloud Service Instance<div dir="ltr" style="text-align: left;" trbidi="on">
In the Final post of this series, we will explore how to Configure Oracle SOA Cloud Service Instance.<br />
<br />
We will be using the configuration details of Oracle Storage Container and Oracle Database Service Instance, created in part1 and part2.<br />
<br />
<b style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, "Palatino Linotype", Palatino, serif; font-size: 15.4px;"><br /></b><b style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, "Palatino Linotype", Palatino, serif; font-size: 15.4px;"><span style="font-size: medium;">Follow the below steps to Configure Oracle SOA Cloud Service Instance:</span></b><br />
<br />
1. Login to My Service Cloud Account.<br />
2. Click on the Menu -> Oracle SOA Cloud Service<br />
3. Click on Create Instance button<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-9j7CLhv8vHo/WGcZfka4ArI/AAAAAAAAcaI/nYaKVhMiyCgOBAokPCR-0Ts40BzSokhLQCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="96" src="https://4.bp.blogspot.com/-9j7CLhv8vHo/WGcZfka4ArI/AAAAAAAAcaI/nYaKVhMiyCgOBAokPCR-0Ts40BzSokhLQCLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
4. Oracle SOA cloud provide multiple options to create the SOA instance i.e. whether we just need Service Bus Instance/SOA instance/Service Bus + SOA Instances etc.<br />
<br />
For this demo, I selected Service Bus + SOA Instance, click Next<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-b94W8piraEY/WGcaWvGCn4I/AAAAAAAAcaQ/QIuuv45Qd00lr9v-KF3NV8DeG-ijh8tSwCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="167" src="https://2.bp.blogspot.com/-b94W8piraEY/WGcaWvGCn4I/AAAAAAAAcaQ/QIuuv45Qd00lr9v-KF3NV8DeG-ijh8tSwCLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
5. Select the version i.e 12.1.3 or 12.2.1<br />
As I am working on 12.1.3, I selected 12.1.3 version. and click Next.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-wcO5UxQ5054/WGcasVOnw7I/AAAAAAAAcaU/SKUxSMlpKAM-6TZ61hLuuEbI7IksEcnTwCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="141" src="https://2.bp.blogspot.com/-wcO5UxQ5054/WGcasVOnw7I/AAAAAAAAcaU/SKUxSMlpKAM-6TZ61hLuuEbI7IksEcnTwCLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
6. Next, we need to enter Instance Details<br />
<b>Instance Configuration:</b><br />
Enter the Instance Name,<br />
Cluster Size<br />
Compute Size i.e. 1 or more manage servers<br />
<b>Weblogic Administrator:</b><br />
Enter weblogic user/password<br />
<b>Database Configuration:</b><br />
Name: Select the Database Cloud Service that we created in part 2<br />
Enter the SYS user<br />
Password<br />
<b>Load Balancer:</b><br />
Provision Load Balancer: YES (If you want to select have a load balancer Instance)<br />
Load Balancer Policy: Least Connection Count<br />
Compute Size:<br />
<b>Storage Container Configuration:</b><br />
Storage Container Name:<br />
Storage User Name:<br />
Password:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-3oATn35PV4Y/WGcdbStdYQI/AAAAAAAAcak/u4t0A9aH0W8LFGxv-h3JuCgCvDGZYye7ACLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="188" src="https://1.bp.blogspot.com/-3oATn35PV4Y/WGcdbStdYQI/AAAAAAAAcak/u4t0A9aH0W8LFGxv-h3JuCgCvDGZYye7ACLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
7. Review All the details and click on Create button.<br />
<a href="https://1.bp.blogspot.com/-fE7xXDDo5gw/WGcd-q_CoPI/AAAAAAAAcas/i-EzOljUdbESQZ797SmPmwPRubnazvBJwCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; display: inline !important; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" height="149" src="https://1.bp.blogspot.com/-fE7xXDDo5gw/WGcd-q_CoPI/AAAAAAAAcas/i-EzOljUdbESQZ797SmPmwPRubnazvBJwCLcB/s320/Capture.JPG" width="320" /></a><br />
<br />
8. It takes around 30 mins to create the SOA instance.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-oaEaiSdECoo/WGcfR3WEq7I/AAAAAAAAca4/w6KNLu7HtbE8P70_-FGjZunYIjrjkae9gCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="75" src="https://3.bp.blogspot.com/-oaEaiSdECoo/WGcfR3WEq7I/AAAAAAAAca4/w6KNLu7HtbE8P70_-FGjZunYIjrjkae9gCLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
9. Once created you can click on action button to access weblogic console/em console/Soa Composer/worklist/service bus etc<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-iX6-F0buxIU/WGclu0eN4NI/AAAAAAAAcbI/AanxBHFylw4A-7st6qQ-J1FNez7YNPceQCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="121" src="https://2.bp.blogspot.com/-iX6-F0buxIU/WGclu0eN4NI/AAAAAAAAcbI/AanxBHFylw4A-7st6qQ-J1FNez7YNPceQCLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Thus, we covered the steps to Provision the Oracle SOA Cloud Service Instance.<br />
<br />
As you can see from the post, it is quite easy to create the SOA instance as compare to the traditional on-premise instances.<br />
We no longer need the activities like Licensing, setting up VM, network configurations, database maintenance, and server maintenance.<br />
<br />
Oracle Cloud also provide the mechanism to configure the client VPN, by which the services deployed on SOA Cloud can access in-house services.<br />
<br />
In further posts, we will cover how to use Cloud SOA instances implementing interfaces.<br />
<br /></div>
Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-80737141080104552072016-12-31T07:56:00.001+05:302016-12-31T07:56:02.828+05:30Getting started with Oracle SOA Cloud Service Part 2: Configure Oracle Database Cloud Service Instance<div dir="ltr" style="text-align: left;" trbidi="on">
In this Post we cover the steps to setup the Oracle Database cloud service instance.<br />
<br />
The Oracle Database Cloud Service instance is like any normal Oracle Database instance that is accessible and managed over a cloud.<br />
<br />
<b>Note: For Oracle SOA cloud service we need to create an Oracle Database Cloud Service with both Cloud Storage and Local Storage backup configuration</b><br />
<b><br /></b>
<b><span style="font-size: large;">Follow the below steps to Configure Oracle Database Cloud Service Instance:</span></b><br />
<br />
1. Log in to your Trial My Services Account<br />
2. From Menu on Left Side, click on Oracle Database Cloud Service<br />
3. Click on <b>Create Service</b> Button<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-tbp7SYCkqnw/WGcOT20aO6I/AAAAAAAAcYc/YG9tdeUOrgYpBc7pI7f_BUJhuL0MpkrgACLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="127" src="https://1.bp.blogspot.com/-tbp7SYCkqnw/WGcOT20aO6I/AAAAAAAAcYc/YG9tdeUOrgYpBc7pI7f_BUJhuL0MpkrgACLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
4. It will ask to either upload the SSH key or generate the new Key. Make sure you save the SSH key as it is needed to connect later.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-otU-yGgj35s/WGcPh5skjMI/AAAAAAAAcYk/DhMDWrujAOQkE6isoRSFgScER79DIaLiACLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="143" src="https://1.bp.blogspot.com/-otU-yGgj35s/WGcPh5skjMI/AAAAAAAAcYk/DhMDWrujAOQkE6isoRSFgScER79DIaLiACLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
5. Next, enter the service name, select the Software Release Version and click Next<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-EPjGpmm-pb8/WGcP5apHlCI/AAAAAAAAcYo/oJA4rFZjLsEsYNGQIRSGBXkCzcGVvFk2gCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="149" src="https://1.bp.blogspot.com/-EPjGpmm-pb8/WGcP5apHlCI/AAAAAAAAcYo/oJA4rFZjLsEsYNGQIRSGBXkCzcGVvFk2gCLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
6. Next, Select the Compute Shape i.e. how many CPU's and RAM is needed.<br />
Since this is a trial I selected 2 CPU with 15GB RAM, which I feel is more than enough.<br />
<br />
Make you select "Both Cloud Storage and Local Storage" Option.<br />
<br />
Here in the container details, we need to provide the container service URL, user/password, which we covered in PART 1.<br />
<br />
If the container does not exist, you can check Create Cloud Storage Container Check Box<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-1PD9D7JComY/WGcQu6MlxqI/AAAAAAAAcY0/iAtIwlBEDXgt9uWmbdXqFX_xETh-cQ5UwCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="146" src="https://1.bp.blogspot.com/-1PD9D7JComY/WGcQu6MlxqI/AAAAAAAAcY0/iAtIwlBEDXgt9uWmbdXqFX_xETh-cQ5UwCLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
7. Review the configuration details, if everything is OK, click on Create Button.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-ag14TY0FmOA/WGcRtaWyDBI/AAAAAAAAcY8/dtHC0b-3Aa8Ti8NejYjnaSmFjklqi9-qACLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="156" src="https://1.bp.blogspot.com/-ag14TY0FmOA/WGcRtaWyDBI/AAAAAAAAcY8/dtHC0b-3Aa8Ti8NejYjnaSmFjklqi9-qACLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
8. It take about 15- 20 mins to create the Database Instance. Till then you will see the status as In progress.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-2pEpttC_XUI/WGcSJ9iClzI/AAAAAAAAcZA/UVR31Lp6TUkp_9fKmnNo_7gQigS2uduJgCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="64" src="https://4.bp.blogspot.com/-2pEpttC_XUI/WGcSJ9iClzI/AAAAAAAAcZA/UVR31Lp6TUkp_9fKmnNo_7gQigS2uduJgCLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
Once the Service is created, you will see the status as completed. Also the action button is enable for the instance.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-lzU5xNPwHGg/WGcSnJUqAwI/AAAAAAAAcZI/52KYqFGMqcYhaT4EhKrOKaX7Af7w3oRWACLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="96" src="https://4.bp.blogspot.com/-lzU5xNPwHGg/WGcSnJUqAwI/AAAAAAAAcZI/52KYqFGMqcYhaT4EhKrOKaX7Af7w3oRWACLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<b><span style="font-size: large;">Next, we will see how to connect to this Oracle Database Cloud Instance over a public network:</span></b><br />
<b><br /></b>
In Order to access the Database Instance over a public network we need to enable the Access Rule.<br />
<br />
1. Identify the Public IP address for our Oracle Database Cloud Instance. To do this, just click on the instance and IP will be visible in Details Section<br />
<br />
2. Click on Action Button, and Click on Access Rules<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-XSUAhmxQQXw/WGcUEeElIkI/AAAAAAAAcZY/tCDZvTHyMVsMFvrYbA78VyuZooBJ7EZqQCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="115" src="https://4.bp.blogspot.com/-XSUAhmxQQXw/WGcUEeElIkI/AAAAAAAAcZY/tCDZvTHyMVsMFvrYbA78VyuZooBJ7EZqQCLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
3. Enable the ora_p2_dblistener access rule<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-BU5n2OMk9ns/WGcUcJXVMKI/AAAAAAAAcZc/SDc3WmvcVB8UdaeKXpISkbWViVH8KWKqACLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="154" src="https://4.bp.blogspot.com/-BU5n2OMk9ns/WGcUcJXVMKI/AAAAAAAAcZc/SDc3WmvcVB8UdaeKXpISkbWViVH8KWKqACLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
4. As the access rule is now enabled, lets connect to the Cloud Database from Local JDeveloper.<br />
Open the JDeveloper, Click on Add Connections under database window<br />
<br />
Provide the required details i.e. User/Password that was enter while creating the Database Instance.<br />
Public IP as a Hostname<br />
SID/Port<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-bBXawnjpy44/WGcVBYGvpvI/AAAAAAAAcZk/dRBLwzIpYv4uYY8if1XhSQ_xlPR2VTm4ACLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="289" src="https://1.bp.blogspot.com/-bBXawnjpy44/WGcVBYGvpvI/AAAAAAAAcZk/dRBLwzIpYv4uYY8if1XhSQ_xlPR2VTm4ACLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
5. If everything is proper, you will see the success message.<br />
<br />
Now you can query the database like any other private database in the network.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-OkCU7cQpRC0/WGcVopEHvfI/AAAAAAAAcZs/Xko-hLDVpDodZNypBB1mO__ICXl3dQutwCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="320" src="https://1.bp.blogspot.com/-OkCU7cQpRC0/WGcVopEHvfI/AAAAAAAAcZs/Xko-hLDVpDodZNypBB1mO__ICXl3dQutwCLcB/s320/Capture.JPG" width="316" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
The Action button provide options to access EM console/DBaas monitor etc, which can be used by the DBA to maintain and manage the DB instances.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-4SHibDS7xMc/WGcWS_J-h5I/AAAAAAAAcZ4/QwbXHa7LZ9Ek8L5PcVDWthpusyXWb6sngCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://3.bp.blogspot.com/-4SHibDS7xMc/WGcWS_J-h5I/AAAAAAAAcZ4/QwbXHa7LZ9Ek8L5PcVDWthpusyXWb6sngCLcB/s1600/Capture.JPG" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Overall, I am pretty impressed with the Oracle Cloud Offering, if you see in this post, I as non-DBA, was able to set up the Database Instance in less than 1 hour with just clicks.<br />
<br />
Next in Part 3, We will explore the steps to Configure Oracle SOA Cloud Service Instance using this DB Instance.<br />
<br />
Ref:<br />
<a href="http://docs.oracle.com/en/database/">http://docs.oracle.com/en/database/</a></div>
Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-8811439350753596202016-12-31T07:08:00.000+05:302016-12-31T07:14:49.799+05:30Getting started with Oracle SOA Cloud Service Part 1: Oracle Storage Cloud Service<div dir="ltr" style="text-align: left;" trbidi="on">
In this Post series, we will cover the steps to configure Oracle SOA Cloud Service.<br />
<br />
Firstly, I signed up for the Oracle Cloud Free Trial Account. The trail account give us 1 month free usage for Oracle Cloud Services.<br />
<br />
To Sign Up, follow the below basic steps:<br />
<br />
<b><a href="https://cloud.oracle.com/">https://cloud.oracle.com</a></b><br />
<br />
Click on Try It<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-Im6_xBVP9aM/WGcCod1e9UI/AAAAAAAAcXs/jpyTRqYUBp4U-S0tk8vtwJdjX-7loYeFwCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="139" src="https://2.bp.blogspot.com/-Im6_xBVP9aM/WGcCod1e9UI/AAAAAAAAcXs/jpyTRqYUBp4U-S0tk8vtwJdjX-7loYeFwCLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Fill in the required details and submit the trail request.<br />
<br />
If the validation goes through, you should get the welcome email which will have the My account details and My Services Details.<br />
<br />
Below are the high steps required to configure Oracle SOA Cloud Service<br />
Step 1: Configure the container in Oracle Storage Cloud Service<br />
Step 2: Configure Oracle Database Cloud Service Instance<br />
Step 3: Configure Oracle SOA Cloud Service Instance<br />
<br />
<br />
<b><span style="font-size: large;">Step 1: Configure the container in Oracle Storage Cloud Service</span></b><br />
<b><span style="font-size: large;"><br /></span></b>
Pre-requisite for Oracle SOA Cloud service is to have the Container Details at the time of creating the SOA cloud Instance.<br />
<br />
Follow the below Steps to create the container in Oracle Storage Cloud Service.<br />
<br />
1. Get the REST URL for Oracle Storage Cloud Service.<br />
<br />
<ul>
<li>Login to My Services URL received in Email.</li>
<li>Click on Storage Link in the Dashboard</li>
</ul>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-8PyKR_bS1Ds/WGcI63EzdoI/AAAAAAAAcX8/I-iYVpH8bHom6vq63s_PYs2FUI3cF3T5QCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="144" src="https://1.bp.blogspot.com/-8PyKR_bS1Ds/WGcI63EzdoI/AAAAAAAAcX8/I-iYVpH8bHom6vq63s_PYs2FUI3cF3T5QCLcB/s320/Capture.JPG" width="320" /></a><a href="https://1.bp.blogspot.com/-8PyKR_bS1Ds/WGcI63EzdoI/AAAAAAAAcX8/I-iYVpH8bHom6vq63s_PYs2FUI3cF3T5QCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><br /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<ul style="text-align: left;">
<li>Under Additional Information you will find the Rest End Point. Copy the URL.</li>
</ul>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-aGhCQr1Y-qM/WGcJmDyZOZI/AAAAAAAAcYE/fBI9Yhe9N8oZcCxcn8FJzLoLh7BTF2jGgCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="130" src="https://1.bp.blogspot.com/-aGhCQr1Y-qM/WGcJmDyZOZI/AAAAAAAAcYE/fBI9Yhe9N8oZcCxcn8FJzLoLh7BTF2jGgCLcB/s320/Capture.JPG" width="320" /></a></div>
<div>
<br /></div>
<br />
<b><span style="font-size: large;"><br /></span></b>
<b><span style="font-size: large;"><br /></span></b>
<b><span style="font-size: large;"><br /></span></b>
<b><span style="font-size: large;"><br /></span></b>
<b><span style="font-size: large;"><br /></span></b>
<b><span style="font-size: large;"><br /></span></b>
2. Step is to Login to Storage Cloud Service.<br />
<br />
For login, refer the below Documentation and click on the respective cloud service URL<br />
<br />
<a href="http://docs.oracle.com/cloud/latest/storagecs_common/CSSTO/GUID-5778ADBB-A0E8-4451-B886-362A3B7237DB.htm#CSSTO-GUID-B431E096-06B5-4FB5-B429-8CE95585BB25">http://docs.oracle.com/cloud/latest/storagecs_common/CSSTO/GUID-5778ADBB-A0E8-4451-B886-362A3B7237DB.htm#CSSTO-GUID-B431E096-06B5-4FB5-B429-8CE95585BB25</a><br />
<br />
The Login Requires the REST endpoint URL and Cloud User/Password.<br />
<br />
Once Login, Click on Create Container to create the storage container for SOA Instance.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-RzXoW89URo8/WGcKq8xVtgI/AAAAAAAAcYQ/Fn9PQnzsV-YEV7KVdPQuzstwl0hy99UywCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="105" src="https://4.bp.blogspot.com/-RzXoW89URo8/WGcKq8xVtgI/AAAAAAAAcYQ/Fn9PQnzsV-YEV7KVdPQuzstwl0hy99UywCLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Make the Note of the Storage Identity Domain.<br />
<br />
At the time of Creating SOA Cloud Instance, you need to form the container URL and provide the login details.<br />
<br />
The container URL is <span style="color: #38761d;">REST URL/</span><span style="color: #660000;">Container Name</span><br />
<br />
For Example: <br />
<span style="color: #38761d;">https://uskpitnewlopes.storage.oraclecloud.com/v1/Storage-uskpitnewlopes</span>/<span style="color: #660000;">soacontainer</span><br />
<br />
Additional Ref:<br />
<a href="http://docs.oracle.com/cloud/latest/storagecs_common/index.html">http://docs.oracle.com/cloud/latest/storagecs_common/index.html</a><br />
<br />
Continue Post in Part 2...</div>
Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-77451418548846984162016-12-27T08:27:00.001+05:302016-12-27T08:29:02.265+05:30Encrypt Sensitive Data in Oracle SOA 12c<div dir="ltr" style="text-align: left;" trbidi="on">
In this post we will explore the option to handle the sensitive data in SOA composite.<br />
<br />
For example, Consider a scenario for creating the employee record which has SSN as one of the element. If we do not apply the encryption for this sensitive data, then we might risk compliance issues as this data might be available in Custom Logs/SOA audit logs and visible to anybody.<br />
<br />
Unlike Oracle SOA 11g, Oracle SOA 12c provide a easy feature to encrypt this sensitive data.<br />
<br />
To demo, we will create a dummy XSD with two elements i.e. Emp Name and SSN.<br />
<br />
The composite simply take the Name/SSN as a input and save the data in table.<br />
<br />
For saving the data in table I just created a DB adaptor. In most of scenario, you might end up calling another system.<br />
<br />
If we run this simple composite by passing some data, and if we look for the soa instance in EM console, the SSN will be clearly visible.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-_GdnS-q8kIA/WGHT_8Jy65I/AAAAAAAAcVc/k_pYjNW0bQ0jbJ8WbdFX_gfRJaLB-UaWwCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="114" src="https://3.bp.blogspot.com/-_GdnS-q8kIA/WGHT_8Jy65I/AAAAAAAAcVc/k_pYjNW0bQ0jbJ8WbdFX_gfRJaLB-UaWwCLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Below are the steps to apply the message encryption from JDEV:<br />
<br />
1. Open the composite in JDEV.<br />
2. Right on the Inbound Partner Link -> Encrypt Sensitive Data<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-_npjOeneRoA/WGHUspZkKqI/AAAAAAAAcVk/EuBomt1ZsngXuK9NtR1ilWHy5M3gxNOLwCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="83" src="https://3.bp.blogspot.com/-_npjOeneRoA/WGHUspZkKqI/AAAAAAAAcVk/EuBomt1ZsngXuK9NtR1ilWHy5M3gxNOLwCLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
3. Click on "oracle/pii_security_policy" and click Edit<br />
4. Set the XPATH for input/output elements which need to be encrypted<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-AjDSvcu0ywQ/WGHVLgunFCI/AAAAAAAAcVs/rvH46YUm3Hg1Z9VVwZ7VuyZIC1PhiPD7wCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="157" src="https://1.bp.blogspot.com/-AjDSvcu0ywQ/WGHVLgunFCI/AAAAAAAAcVs/rvH46YUm3Hg1Z9VVwZ7VuyZIC1PhiPD7wCLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
5. You might consider cloning the policy if it need to be used in Multiple composites with different parameters.<br />
<br />
Note: The clone policy cannot be configured from JDEV, we need to configure it from EM console. <br />
If you need to override clone properties at runtime, we can do via WLST scripts. Please refer below documentation.<br />
Ref: (Configuring Secure Conversation Using WLST)<br />
https://docs.oracle.com/middleware/1213/owsm/security/configure-secureconv.htm#OWSMS5360<br />
<br />
6. To configure the CSF key on your environment. By default the CSF key is "pii-csf-key".<br />
You can add the CSF key either from EM console or from Jdev.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-ZjlD-wrOCv8/WGHWgRkkQoI/AAAAAAAAcWE/Pg16JtwNVLAKi5GIyqU4c66D9srFhjFCQCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="73" src="https://4.bp.blogspot.com/-ZjlD-wrOCv8/WGHWgRkkQoI/AAAAAAAAcWE/Pg16JtwNVLAKi5GIyqU4c66D9srFhjFCQCLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
If the key is not present on your server then you will get below error.<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span><b>InvalidSecurity : error in processing the WS-Security security header</b><br />
<div>
<br /></div>
Once you deploy this composite, and if we retest, we should see that the SSN element is encrypted.<br />
<a href="https://1.bp.blogspot.com/-4C93PxF1dZ8/WGHXH_Y9Q9I/AAAAAAAAcWI/ghRjCjxnXeE3Jnnut94fjk-7GOQs5AhDQCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; display: inline !important; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" height="86" src="https://1.bp.blogspot.com/-4C93PxF1dZ8/WGHXH_Y9Q9I/AAAAAAAAcWI/ghRjCjxnXeE3Jnnut94fjk-7GOQs5AhDQCLcB/s320/Capture.JPG" width="320" /></a><br />
<br />
<b>Note: </b>As the current policy, the Encrypted elements need to be Decrypted before leaving the current composite i.e. another composite will not be able to decrypted it.<br />
<br />
Thus in this example, if I do not drcrypt the SSN then in Database it will be stored in encrypted format which will be of no use.<br />
<br />
To configure decryption:<br />
7. Right Click on outbound partner link, and you should see Decrypt Sensitive Data option<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-rMiWgUuFbNs/WGHYH3tLGXI/AAAAAAAAcWY/mVYHVPTBafglNUJ-w4Wy5As_N749NKWCgCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="74" src="https://4.bp.blogspot.com/-rMiWgUuFbNs/WGHYH3tLGXI/AAAAAAAAcWY/mVYHVPTBafglNUJ-w4Wy5As_N749NKWCgCLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
8. Select the element you want to decrypt, this time it will be the transformed element.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-f9mfYCXMIFo/WGHYaU0ZfbI/AAAAAAAAcWc/XDtaQ5DFaXoPHL0lC_ERlHS3W3diKOygwCLcB/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="111" src="https://3.bp.blogspot.com/-f9mfYCXMIFo/WGHYaU0ZfbI/AAAAAAAAcWc/XDtaQ5DFaXoPHL0lC_ERlHS3W3diKOygwCLcB/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Thus as you can see, it is much easier to implement message level security as compared to 11g.<br />
<br />
<br /></div>
Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-60881723450163875772016-03-13T17:32:00.000+05:302016-03-13T17:32:04.036+05:30Passing/Receiving HTTP header in SOA BPEL<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="WordSection1">
<div class="MsoNormal">
In this example we will explore how to pass or receive the parameters from HTTP header.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
We will explore below two scenarios, <o:p></o:p></div>
<div class="MsoNormal">
To receive the HTTP header from SOAP request.<o:p></o:p></div>
<div class="MsoNormal">
To pass the HTTP header while calling soap service.<o:p></o:p></div>
<div class="MsoNormal">
SOA version used: 12.1.3c<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b>To receive the HTTP header from SOAP request:<o:p></o:p></b></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
First we need to add the property “oracle.webservices.http.headers” under binding.ws in Service element in composite.xml. This property contains the comma separated parameters that we are expecting in request. <o:p></o:p></div>
<div class="MsoNormal">
Example, ENV,VERSION.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<binding.ws port="http://xmlns.oracle.com/SOA_Learning/HTTPHeader/HTTPHeaderTest#wsdl.endpoint(httpheadertest_client_ep/HTTPHeaderTest_pt)"><o:p></o:p></div>
<div class="MsoNormal">
<b> <property name="oracle.webservices.http.headers">ENV,VERSION</property><o:p></o:p></b></div>
<div class="MsoNormal">
</binding.ws><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Now in BPEL source under receive activity, we need add <fromProperties> element which will copy the parameters from HTTP header to BPEL variable.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<receive name="receiveInput" partnerLink="httpheadertest_client" portType="client:HTTPHeaderTest" operation="process" variable="inputVariable" createInstance="yes"><o:p></o:p></div>
<div class="MsoNormal">
<bpelx:fromProperties ><o:p></o:p></div>
<div class="MsoNormal">
<b> <bpelx:fromProperty name=" ENV" variable="var_ENV"></bpelx:fromProperty><o:p></o:p></b></div>
<div class="MsoNormal">
<b> <bpelx:fromProperty name=" VERSION" variable="var_VERSION"></bpelx:fromProperty><o:p></o:p></b></div>
<div class="MsoNormal">
</bpelx:fromProperties><o:p></o:p></div>
<div class="MsoNormal">
</receive><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Create two variables name “var_ENV” and “var_VERSION” of type string.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<a href="http://3.bp.blogspot.com/-Ef7uXxj8Wc0/VuVWY6CEsxI/AAAAAAAAWWA/5gqRphxZQhg2S3Hrb-77yRRUmWrgz2ilg/s1600/image002-734293.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6261505842823344914" src="https://3.bp.blogspot.com/-Ef7uXxj8Wc0/VuVWY6CEsxI/AAAAAAAAWWA/5gqRphxZQhg2S3Hrb-77yRRUmWrgz2ilg/s320/image002-734293.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Now for DEMO purpose we will return the parameters from HTTP header to SOAP response.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<a href="http://3.bp.blogspot.com/-yoo3P02_rZU/VuVWZYaaWGI/AAAAAAAAWWI/Dg21NPJNqrsNP7OYo_2WphW2yDPDx4NZw/s1600/image001-736559.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6261505850978490466" src="https://3.bp.blogspot.com/-yoo3P02_rZU/VuVWZYaaWGI/AAAAAAAAWWI/Dg21NPJNqrsNP7OYo_2WphW2yDPDx4NZw/s320/image001-736559.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Now we can test our service from EM or SOAP UI. EM console also allow use to set the HTTP header in request.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<a href="http://1.bp.blogspot.com/-zZ92-vHthtE/VuVWa9ya-CI/AAAAAAAAWWQ/VMidkTjeVtshpqqOHfNaIfDSx2Q73_kKA/s1600/image003-742501.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6261505878191175714" src="https://1.bp.blogspot.com/-zZ92-vHthtE/VuVWa9ya-CI/AAAAAAAAWWQ/VMidkTjeVtshpqqOHfNaIfDSx2Q73_kKA/s320/image003-742501.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
RESPONSE:<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<a href="http://1.bp.blogspot.com/-wMK_CsYc4hg/VuVWbew4sDI/AAAAAAAAWWY/a1B7Pcb7BSYeEoUYo4tmO4sYMUOz2QI9w/s1600/image004-744535.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6261505887043104818" src="https://1.bp.blogspot.com/-wMK_CsYc4hg/VuVWbew4sDI/AAAAAAAAWWY/a1B7Pcb7BSYeEoUYo4tmO4sYMUOz2QI9w/s320/image004-744535.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
If we see EM console, we will find below trace logs:<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<table border="0" cellpadding="0" cellspacing="0" class="MsoNormalTable" style="background: white;" summary="Audit Trail"> <tbody>
<tr> <td nowrap="" style="padding: 0in 0in 0in 0in;"><div class="MsoNormal">
<b><span style="color: black; font-family: "tahoma" , sans-serif; font-size: 8.5pt;">receiveInput</span></b><span style="color: #333333; font-family: "tahoma" , sans-serif; font-size: 8.5pt;"><o:p></o:p></span></div>
</td> </tr>
<tr> <td nowrap="" style="padding: 0in 0in 0in 0in;"><div class="MsoNormal">
<span style="color: #999999; font-family: "verdana" , sans-serif; font-size: 8.5pt;"> Mar 13, 2016 5:18:05 PM </span><span style="color: #333333; font-family: "tahoma" , sans-serif; font-size: 8.5pt;"><a href="http://2.bp.blogspot.com/--I4hcvhc5-I/VuVWb9kPYjI/AAAAAAAAWWg/Fng_3GRfv5wzHBAlvDIpJvU6rGoik7PGw/s1600/image005-746739.gif"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6261505895311565362" src="https://2.bp.blogspot.com/--I4hcvhc5-I/VuVWb9kPYjI/AAAAAAAAWWg/Fng_3GRfv5wzHBAlvDIpJvU6rGoik7PGw/s320/image005-746739.gif" /></a></span><b><span style="color: black; font-family: "tahoma" , sans-serif; font-size: 8.5pt;">Received property "VERSION", value is "1.0".</span></b><span style="color: #333333; font-family: "tahoma" , sans-serif; font-size: 8.5pt;"><o:p></o:p></span></div>
</td> </tr>
<tr> <td nowrap="" style="padding: 0in 0in 0in 0in;"><div class="MsoNormal">
<span style="color: #999999; font-family: "verdana" , sans-serif; font-size: 8.5pt;"> Mar 13, 2016 5:18:05 PM </span><span style="color: #333333; font-family: "tahoma" , sans-serif; font-size: 8.5pt;"><a href="http://2.bp.blogspot.com/--I4hcvhc5-I/VuVWb9kPYjI/AAAAAAAAWWg/Fng_3GRfv5wzHBAlvDIpJvU6rGoik7PGw/s1600/image005-746739.gif"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6261505895311565362" src="https://2.bp.blogspot.com/--I4hcvhc5-I/VuVWb9kPYjI/AAAAAAAAWWg/Fng_3GRfv5wzHBAlvDIpJvU6rGoik7PGw/s320/image005-746739.gif" /></a></span><b><span style="color: black; font-family: "tahoma" , sans-serif; font-size: 8.5pt;">Received property "ENV", value is "DEV".</span></b><span style="color: #333333; font-family: "tahoma" , sans-serif; font-size: 8.5pt;"><o:p></o:p></span></div>
</td> </tr>
<tr> <td nowrap="" style="padding: 0in 0in 0in 0in;"><div class="MsoNormal">
<span style="border: dotted #003286 1.0pt; color: #999999; font-family: "verdana" , sans-serif; font-size: 8.5pt; padding: 0in;"> Mar 13, 2016 5:18:05 PM </span><span style="color: #333333; font-family: "tahoma" , sans-serif; font-size: 8.5pt;"><a href="http://2.bp.blogspot.com/--I4hcvhc5-I/VuVWb9kPYjI/AAAAAAAAWWg/Fng_3GRfv5wzHBAlvDIpJvU6rGoik7PGw/s1600/image005-746739.gif"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6261505895311565362" src="https://2.bp.blogspot.com/--I4hcvhc5-I/VuVWb9kPYjI/AAAAAAAAWWg/Fng_3GRfv5wzHBAlvDIpJvU6rGoik7PGw/s320/image005-746739.gif" /></a></span><span style="border: dotted #003286 1.0pt; color: black; font-family: "tahoma" , sans-serif; font-size: 8.5pt; padding: 0in;">Received "process" call from partner "httpheadertest_client"</span><span style="color: #333333; font-family: "tahoma" , sans-serif; font-size: 8.5pt;"><o:p></o:p></span></div>
</td> </tr>
</tbody> </table>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Similarly, for passing the parameters in HTTP Header, we need to add,<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b> <property name="oracle.webservices.http.headers">ENV,VERSION</property><o:p></o:p></b></div>
<div class="MsoNormal">
To binding.ws under reference tag in composite.xml.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
In INVOKE activity we need to use toProperties element.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<bpelx:toProperties ><o:p></o:p></div>
<div class="MsoNormal">
<bpelx:toProperty name="ENV" variable="var_ENV">"lllf"</bpelx:toProperty><o:p></o:p></div>
<div class="MsoNormal">
<bpelx:toProperty name="VERSION" variable="ver_VERSION"></bpelx:toProperty><o:p></o:p></div>
<div class="MsoNormal">
</bpelx:toProperties><o:p></o:p></div>
<div class="MsoNormal">
<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Note:<o:p></o:p></div>
<div class="MsoNormal">
The above properties does not work for REPLY activity as SOA does not support returning the parameter in HTTP header.<o:p></o:p></div>
<div class="MsoNormal">
Ref: <a href="https://community.oracle.com/thread/3907803">https://community.oracle.com/thread/3907803</a><o:p></o:p></div>
<div class="MsoNormal">
Ref: <a href="http://biemond.blogspot.in/2012/04/retrieve-or-set-http-header-from-oracle.html"> http://biemond.blogspot.in/2012/04/retrieve-or-set-http-header-from-oracle.html</a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Thanks.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
</div>
</div>
Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-3209618734077553162016-03-13T12:16:00.000+05:302016-03-13T12:16:44.916+05:30Configuring REST service in SOA using HTTP Adapter - GET/POST<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="WordSection1">
<div class="MsoNormal">
In this example, we will explore how to configure REST service in SOA for both GET and POST request.<o:p></o:p></div>
<div class="MsoNormal">
For this example, we will create a XSD which takes firstname and lastname as input and returns Full Name as output.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
First, we will create a new project call, RestNamingService.<o:p></o:p></div>
<div class="MsoNormal">
<a href="http://1.bp.blogspot.com/-Gm_xGncd69E/VuUMj2vO4YI/AAAAAAAAWTc/GPyWqPSpeEIHb-Jw_U9ZFrC35ZCcDtKEA/s1600/image001-735339.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6261424667057119618" src="https://1.bp.blogspot.com/-Gm_xGncd69E/VuUMj2vO4YI/AAAAAAAAWTc/GPyWqPSpeEIHb-Jw_U9ZFrC35ZCcDtKEA/s320/image001-735339.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
Our XSD will look like below:<o:p></o:p></div>
<div class="MsoNormal">
<a href="http://3.bp.blogspot.com/-zK9d7tvHQss/VuUMklMpfrI/AAAAAAAAWTk/FUqLQJ3w0rA5x6yj7A1hseOPMbEBMDxLw/s1600/image002-737538.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6261424679528529586" src="https://3.bp.blogspot.com/-zK9d7tvHQss/VuUMklMpfrI/AAAAAAAAWTk/FUqLQJ3w0rA5x6yj7A1hseOPMbEBMDxLw/s320/image002-737538.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
<?xml version="1.0" encoding="windows-1252" ?><o:p></o:p></div>
<div class="MsoNormal">
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://www.example.org"<o:p></o:p></div>
<div class="MsoNormal">
targetNamespace="http://www.example.org" elementFormDefault="qualified"><o:p></o:p></div>
<div class="MsoNormal">
<xsd:element name="request"><o:p></o:p></div>
<div class="MsoNormal">
<xsd:annotation><o:p></o:p></div>
<div class="MsoNormal">
<xsd:documentation>A sample element</xsd:documentation><o:p></o:p></div>
<div class="MsoNormal">
</xsd:annotation><o:p></o:p></div>
<div class="MsoNormal">
<xsd:complexType><o:p></o:p></div>
<div class="MsoNormal">
<xsd:sequence><o:p></o:p></div>
<div class="MsoNormal">
<xsd:element name="firstname" type="xsd:string"/><o:p></o:p></div>
<div class="MsoNormal">
<xsd:element name="lastname" type="xsd:string"/><o:p></o:p></div>
<div class="MsoNormal">
</xsd:sequence><o:p></o:p></div>
<div class="MsoNormal">
</xsd:complexType><o:p></o:p></div>
<div class="MsoNormal">
</xsd:element><o:p></o:p></div>
<div class="MsoNormal">
<xsd:element name="response"><o:p></o:p></div>
<div class="MsoNormal">
<xsd:complexType><o:p></o:p></div>
<div class="MsoNormal">
<xsd:sequence><o:p></o:p></div>
<div class="MsoNormal">
<xsd:element name="fullname" type="xsd:string"/><o:p></o:p></div>
<div class="MsoNormal">
</xsd:sequence><o:p></o:p></div>
<div class="MsoNormal">
</xsd:complexType><o:p></o:p></div>
<div class="MsoNormal">
</xsd:element><o:p></o:p></div>
<div class="MsoNormal">
</xsd:schema><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Now drag HTTP binding to service end of Composite.xml. Name service: <b>httpserviceGET</b><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<a href="http://1.bp.blogspot.com/-vwIRBL7FaFs/VuUMk7Bm0ZI/AAAAAAAAWTs/AAoNFR5qRo87P9ZJU4PRHC--s7EA3EcKQ/s1600/image003-739337.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6261424685387796882" src="https://1.bp.blogspot.com/-vwIRBL7FaFs/VuUMk7Bm0ZI/AAAAAAAAWTs/AAoNFR5qRo87P9ZJU4PRHC--s7EA3EcKQ/s320/image003-739337.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<a href="http://1.bp.blogspot.com/-otIWUOyDQgo/VuUMlPEpptI/AAAAAAAAWT0/gVWHdoTAiP4qeC8JV7jSych6tVvT5lonw/s1600/image004-740687.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6261424690769274578" src="https://1.bp.blogspot.com/-otIWUOyDQgo/VuUMlPEpptI/AAAAAAAAWT0/gVWHdoTAiP4qeC8JV7jSych6tVvT5lonw/s320/image004-740687.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Since we are expecting the response, we will select request-response pattern. The request type is GET.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<a href="http://4.bp.blogspot.com/-IB2hPi_vya0/VuUMljsuUFI/AAAAAAAAWT8/m80ZU4iHUZAAFTtLTO9uMlCo3hPlduHCw/s1600/image005-741949.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6261424696306061394" src="https://4.bp.blogspot.com/-IB2hPi_vya0/VuUMljsuUFI/AAAAAAAAWT8/m80ZU4iHUZAAFTtLTO9uMlCo3hPlduHCw/s320/image005-741949.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Select the request and response element defined in XSD.<o:p></o:p></div>
<div class="MsoNormal">
<a href="http://3.bp.blogspot.com/-vAn_cyJYDxA/VuUMlxbogtI/AAAAAAAAWUE/1kG3oQQavuU997BEaUJSgsTa6NLG-fz2g/s1600/image006-743407.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6261424699992474322" src="https://3.bp.blogspot.com/-vAn_cyJYDxA/VuUMlxbogtI/AAAAAAAAWUE/1kG3oQQavuU997BEaUJSgsTa6NLG-fz2g/s320/image006-743407.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Click Finish.<o:p></o:p></div>
<div class="MsoNormal">
This will generate the service binding name httpserviceGET.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Now Drag the BPEL component in Composite. Name it as processGet.<o:p></o:p></div>
<div class="MsoNormal">
<a href="http://4.bp.blogspot.com/-CjWBdI6bRuY/VuUMmNNAynI/AAAAAAAAWUM/rXj9RYRUa-oWCqGAWYxQdmRTrinPNjpZQ/s1600/image007-744746.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6261424707447343730" src="https://4.bp.blogspot.com/-CjWBdI6bRuY/VuUMmNNAynI/AAAAAAAAWUM/rXj9RYRUa-oWCqGAWYxQdmRTrinPNjpZQ/s320/image007-744746.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Wire the httpserviceGet and processGet component.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Add receive and reply activity in processGet bpel service.<o:p></o:p></div>
<div class="MsoNormal">
<a href="http://1.bp.blogspot.com/-bFbVu3EltSs/VuUMmsRz0NI/AAAAAAAAWUU/STBA2hdhXkwknZpo5YMJ5ENGPw2newyCQ/s1600/image008-745993.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6261424715788964050" src="https://1.bp.blogspot.com/-bFbVu3EltSs/VuUMmsRz0NI/AAAAAAAAWUU/STBA2hdhXkwknZpo5YMJ5ENGPw2newyCQ/s320/image008-745993.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Add assign activity to concat the firstname and lastname and set it to response element.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<a href="http://1.bp.blogspot.com/-QWGi4c0MaF0/VuUMm4kK_AI/AAAAAAAAWUc/bPzL7i3sIhUHqN_GhTAZ355Dr8KrvUMLQ/s1600/image009-747211.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6261424719087205378" src="https://1.bp.blogspot.com/-QWGi4c0MaF0/VuUMm4kK_AI/AAAAAAAAWUc/bPzL7i3sIhUHqN_GhTAZ355Dr8KrvUMLQ/s320/image009-747211.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Now deploy the RestNamingService project to SOA server.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Testing:<o:p></o:p></div>
<div class="MsoNormal">
We can test our service by passing the parameters in request URL.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<a href="http://localhost:7101/soa-infra/services/default/RestNamingService/httpServiceGET?firstname=John&lastname=smith&operationName=Request-Response">http://localhost:7101/soa-infra/services/default/RestNamingService/httpServiceGET?<span style="color: red;">firstname=John&lastname=smith&operationName=Request-Response</span></a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Response:<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="color: black; font-family: "courier new"; font-size: 10.0pt;"><response xmlns="http://www.example.org"><o:p></o:p></span></div>
<div class="MsoNormal">
<span style="color: black; font-family: "courier new"; font-size: 10.0pt;"><fullname xmlns="http://www.example.org">John smith</fullname><o:p></o:p></span></div>
<div class="MsoNormal">
<span style="color: black; font-family: "courier new"; font-size: 10.0pt;"></response><o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Post Service:<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
For testing the POST service we will modify existing service by changing the pattern to POST.<o:p></o:p></div>
<div class="MsoNormal">
Double click the httpserviceGet service endpoint in composite xml and change the pattern to POST.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<a href="http://4.bp.blogspot.com/-gMFkV8OLON0/VuUMnLUwBMI/AAAAAAAAWUk/R4Cbjr2h4W4qL41cEvz9tIWC63zOSDSxA/s1600/image010-748614.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6261424724122797250" src="https://4.bp.blogspot.com/-gMFkV8OLON0/VuUMnLUwBMI/AAAAAAAAWUk/R4Cbjr2h4W4qL41cEvz9tIWC63zOSDSxA/s320/image010-748614.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Now reploy the project on SOA server.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
In order to test the POST service we will create a REST project in SOAP UI.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<a href="http://2.bp.blogspot.com/-gVxrjFt7xPY/VuUMn-8jy1I/AAAAAAAAWUs/WTaMulHjBFkkSYNYcNTxYWEoRwESFOyuQ/s1600/image011-750875.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6261424737979976530" src="https://2.bp.blogspot.com/-gVxrjFt7xPY/VuUMn-8jy1I/AAAAAAAAWUs/WTaMulHjBFkkSYNYcNTxYWEoRwESFOyuQ/s320/image011-750875.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
This how we can create REST service in SOA using http binding.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
</div>
</div>
Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-73467467521088131692016-03-09T13:49:00.000+05:302016-03-09T13:49:27.578+05:30Oracle SOA suite Packaging<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="WordSection1">
<div class="MsoNormal">
In this post we will go through some basic steps to automate the daily tasks i.e. compilation and deployment of SOA projects.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
In order to deploy the SOA projects on server we usually follow below two approaches:<o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
1. Compile and Deploy using JDEV<o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
2. Compile and Export the SAR using JDEV. Deploy using EM console.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
The above is adopted in development phase. In testing and production we need to automate this deployment activities in order to avoid any human error. SOA suite provide us the ANT scripts to perform this tasks.<o:p></o:p></div>
<div class="MsoNormal">
This same scripts are used by JDEV to perform the task like deploy/compile etc.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
We will now go show the steps to use some basic commands: <o:p> </o:p></div>
<div class="MsoNormal">
I am using Oracle SOA 12c environment.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Pre-requisites:</div>
<div class="MsoNormal">
Configure the ANT_HOME, JAVA_HOME and PATH variables. </div>
<div class="MsoNormal">
1. ANT_HOME = (MIDDLEWARE_HOME)\Middleware\Oracle_Home\oracle_common\modules\org.apache.ant_1.9.2. PATH=%PATH%;(MIDDLEWARE_HOME)\Middleware\Oracle_Home\oracle_common\modules\org.apache.ant_1.9.2\bin</div>
<div class="MsoNormal">
<o:p></o:p></div>
<div class="MsoNormal">
3. JAVA_HOME=C:\Program Files\Java\jdk1.7.0_71<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b>The SOA build scripts files can be found under (Middlewarehome)\Middleware\Oracle_Home\soa\bin<o:p></o:p></b></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoListParagraph" style="mso-list: l1 level1 lfo2; text-indent: -.25in;">
<b><span style="mso-list: Ignore;">1.<span style="font: 7.0pt "Times New Roman";"> </span></span></b><b>Compiling the SOA project: ant-sca-compile.xml<o:p></o:p></b></div>
<div class="MsoListParagraph">
This script is used to compile the SOA project. It takes composite.xml path and the output file path. This compilation output is written to output file in XML format.<o:p></o:p></div>
<div class="MsoListParagraph">
<br /></div>
<div class="MsoListParagraph">
ant -buildfile ant-sca-compile.xml -Dscac.input=C:\JDeveloper\mywork\SOA_Learning\Flow\SOA\composite.xml -Dscac.output=D:\tmp\SOA\compileoutput.txt<o:p></o:p></div>
<div class="MsoNormal">
<a href="http://3.bp.blogspot.com/-KsyZNBZL2Og/Vt_ZlBh1IjI/AAAAAAAAWOw/FwpHxglLzIg/s1600/image001-703377.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6259961237156995634" src="https://3.bp.blogspot.com/-KsyZNBZL2Og/Vt_ZlBh1IjI/AAAAAAAAWOw/FwpHxglLzIg/s320/image001-703377.png" /></a><o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l1 level1 lfo2; text-indent: -.25in;">
2. <b>Generate .SAR file as .JAR: ant-sca-package.xml<o:p></o:p></b></div>
<div class="MsoListParagraph">
This script is used to deploy the SOA project as .SAR called .JAR file. <o:p></o:p></div>
<div class="MsoListParagraph">
<br /></div>
<div class="MsoListParagraph">
ant -buildfile ant-sca-package.xml -DcompositeDir=C:\JDeveloper\mywork\SOA_Learning\Flow\SOA -DcompositeName=Flow -Drevision=1.2<o:p></o:p></div>
<div class="MsoListParagraph">
<a href="http://1.bp.blogspot.com/-8wQiEu77V7g/Vt_ZlVIK73I/AAAAAAAAWO4/XvfTKI62PXk/s1600/image002-705239.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6259961242418081650" src="https://1.bp.blogspot.com/-8wQiEu77V7g/Vt_ZlVIK73I/AAAAAAAAWO4/XvfTKI62PXk/s320/image002-705239.png" /></a><o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l1 level1 lfo2; text-indent: -.25in;">
3. <b>Deploy to SOA Server: ant-sca-deploy.xml</b><o:p></o:p></div>
<div class="MsoListParagraph">
This script is used to deploy the generated .JAR to SOA Server. We can set the configuration plan based on the environment on which we are deploying i.e. DEV/UAT/PROD<o:p></o:p></div>
<div class="MsoListParagraph">
We can use the same script to perform following activities i.e. undeploy/ exportComposite/ exportUpdates/ importUpdates/ exportSharedData/ undeployRetiredComposites<o:p></o:p></div>
<div class="MsoListParagraph">
<br /></div>
<div class="MsoListParagraph">
ant -buildfile ant-sca-deploy.xml -DserverURL=http://localhost:7101 -DsarLocation=C:\JDeveloper\mywork\SOA_Learning\Flow\SOA\deploy\sca_Flow_rev1.1.jar -Doverwrite=true -Duser=weblogic -Dpassword=welcome1 -DforceDefault=true -DkeepInstancesOnRedeploy=false -DregenerateRuleBase=true -Dconfigplan=C:\JDeveloper\mywork\SOA_Learning\Flow\Flow_cfgplan.xml -DfailOnError=true<o:p></o:p></div>
<div class="MsoListParagraph">
<a href="http://2.bp.blogspot.com/-O3fvnAvO8DU/Vt_ZlrsmufI/AAAAAAAAWPA/Yw-J_HQTHFE/s1600/image003-706442.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6259961248476477938" src="https://2.bp.blogspot.com/-O3fvnAvO8DU/Vt_ZlrsmufI/AAAAAAAAWPA/Yw-J_HQTHFE/s320/image003-706442.png" /></a><o:p></o:p></div>
<div class="MsoListParagraph">
<br /></div>
<div class="MsoNormal">
The above commands can be configured with Hudson to implement continuous integration.<o:p></o:p></div>
</div>
<br /></div>
Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-7310630775062430272016-03-03T00:33:00.000+05:302016-03-06T12:33:35.447+05:30Oracle BPM 12c Process Asset Manager Feature<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="WordSection1">
<div class="MsoNormal">
In this post we will cover Oracle BPM 12c feature called PAM i.e. Process Asset Manager.<o:p></o:p></div>
<div class="MsoNormal">
The PAM allow seamless collaboration between multiple participants in BPM like Business Analyst and BPM developer.<o:p></o:p></div>
<div class="MsoNormal">
It allow us to share the BPM artifacts seamlessly between BPM process composer and BPM Studio (JDEV). <o:p></o:p></div>
<div class="MsoNormal">
The artifacts are maintain in in-built SVN.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
To illustrate in more details let’s take an example:<o:p></o:p></div>
<div class="MsoNormal">
As a part of BPM process analysis and implementation, Business Analyst design the process in BPM composer. Once the process is completed from BA, it is given to Developer for implementation.<o:p></o:p></div>
<div class="MsoNormal">
During implementation, there can be multiple to and from between Developer and the BA. Thus the latest BPM artifact need to be available to both users i.e. BPM composer users and BPM studio users.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
In below example we will see how the process created in BPM composer can be shared with BPM studio user using PAM.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoListParagraph" style="mso-list: l2 level1 lfo2; text-indent: -.25in;">
1. Login to BPM composer <o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l2 level1 lfo2; text-indent: -.25in;">
2. Create a new space “order”<o:p></o:p></div>
<div class="MsoListParagraph">
<a href="http://4.bp.blogspot.com/-_sobJkeBluI/Vtbc9lRxBUI/AAAAAAAAWII/xNj89Yvc3rg/s1600/image001-745828.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6257431682814641474" src="https://4.bp.blogspot.com/-_sobJkeBluI/Vtbc9lRxBUI/AAAAAAAAWII/xNj89Yvc3rg/s320/image001-745828.png" /></a><o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l2 level1 lfo2; text-indent: -.25in;">
2. Click on Order Space.<o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l2 level1 lfo2; text-indent: -.25in;">
4. Create a new BPM project called OrderProcessing<o:p></o:p></div>
<div class="MsoListParagraph">
<a href="http://1.bp.blogspot.com/-dWBLqzlSc0g/Vtbc93khIuI/AAAAAAAAWIQ/m9ceMMadjvY/s1600/image002-747802.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6257431687725130466" src="https://1.bp.blogspot.com/-dWBLqzlSc0g/Vtbc93khIuI/AAAAAAAAWIQ/m9ceMMadjvY/s320/image002-747802.png" /></a><o:p></o:p></div>
<div class="MsoListParagraph">
<br /></div>
<div class="MsoListParagraph" style="mso-list: l2 level1 lfo2; text-indent: -.25in;">
5. For demo, Create a simple process with an abstract activity. Save the process.<o:p></o:p></div>
<div class="MsoListParagraph">
<a href="http://4.bp.blogspot.com/-XBz3TrgIBPA/Vtbc-fYaaKI/AAAAAAAAWIY/2F7_MfVRawU/s1600/image003-749016.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6257431698411776162" src="https://4.bp.blogspot.com/-XBz3TrgIBPA/Vtbc-fYaaKI/AAAAAAAAWIY/2F7_MfVRawU/s320/image003-749016.png" /></a><o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l2 level1 lfo2; text-indent: -.25in;">
6. Once saved, you will see the once to publish the project.<o:p></o:p></div>
<div class="MsoListParagraph">
<a href="http://4.bp.blogspot.com/-IFuMXPvdOdc/Vtbc-jolrDI/AAAAAAAAWIg/cKFPEBdAbRU/s1600/image004-750289.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6257431699553365042" src="https://4.bp.blogspot.com/-IFuMXPvdOdc/Vtbc-jolrDI/AAAAAAAAWIg/cKFPEBdAbRU/s320/image004-750289.png" /></a><o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l2 level1 lfo2; text-indent: -.25in;">
7. Click on publish, add relevant comments, and click on Publish.<o:p></o:p></div>
<div class="MsoListParagraph">
<b>Note: when you publish, the changes get checked-in in SVN<o:p></o:p></b></div>
<div class="MsoListParagraph">
<a href="http://1.bp.blogspot.com/--51kIS3kbPk/Vtbc-zVNU_I/AAAAAAAAWIo/pv2gy4JU57A/s1600/image005-751387.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6257431703767045106" src="https://1.bp.blogspot.com/--51kIS3kbPk/Vtbc-zVNU_I/AAAAAAAAWIo/pv2gy4JU57A/s320/image005-751387.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Now say the BPM developer want to implement above designed process. He need to get the process in BPM studio. We can use PAM to checkout the project in JDEV.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoListParagraph" style="mso-list: l2 level1 lfo2; text-indent: -.25in;">
8. Open JDEV. Create new BPM application.<o:p></o:p></div>
<div class="MsoListParagraph">
<a href="http://1.bp.blogspot.com/-yRChl49DHTI/Vtbc_ILEkXI/AAAAAAAAWIw/Fv1HsbRCBbU/s1600/image006-752514.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6257431709361672562" src="https://1.bp.blogspot.com/-yRChl49DHTI/Vtbc_ILEkXI/AAAAAAAAWIw/Fv1HsbRCBbU/s320/image006-752514.png" /></a><o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l2 level1 lfo2; text-indent: -.25in;">
9. Now we need to create PAM connection. <o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l2 level1 lfo2; text-indent: -.25in;">
10. Click on Window->Process Asset Manager Navigator<o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l2 level1 lfo2; text-indent: -.25in;">
11. Right Click PAM->New PAM connection<o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l2 level1 lfo2; text-indent: -.25in;">
12. Create the PAM connection to BPM server on which the we create the BPM project using BPM composer.<o:p></o:p></div>
<div class="MsoListParagraph">
<a href="http://3.bp.blogspot.com/-kvMN_uxreWc/Vtbc_fD1u3I/AAAAAAAAWI4/tfNQHZRsb1E/s1600/image007-753666.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6257431715505355634" src="https://3.bp.blogspot.com/-kvMN_uxreWc/Vtbc_fD1u3I/AAAAAAAAWI4/tfNQHZRsb1E/s320/image007-753666.png" /></a><o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l2 level1 lfo2; text-indent: -.25in;">
13. You will be able to see the Order Space that we created in Composer.<o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l2 level1 lfo2; text-indent: -.25in;">
14. Expand Order Space. Right Click the OrderProcessing Process and Click CheckOut. Select Destination Application and click OK.<o:p></o:p></div>
<div class="MsoListParagraph">
<a href="http://3.bp.blogspot.com/-8XHGQR20HDg/Vtbc_wC_gPI/AAAAAAAAWJA/r9Jx09mMQRo/s1600/image008-754874.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6257431720065204466" src="https://3.bp.blogspot.com/-8XHGQR20HDg/Vtbc_wC_gPI/AAAAAAAAWJA/r9Jx09mMQRo/s320/image008-754874.png" /></a><o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l2 level1 lfo2; text-indent: -.25in;">
15. Now you will be able to see the OrderProcessing Project under Application in JDEV.<o:p></o:p></div>
<div class="MsoListParagraph">
<a href="http://4.bp.blogspot.com/-RVuxQbsyjQA/VtbdAFxjaJI/AAAAAAAAWJI/-OB8Wgdk6I8/s1600/image009-756125.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6257431725897640082" src="https://4.bp.blogspot.com/-RVuxQbsyjQA/VtbdAFxjaJI/AAAAAAAAWJI/-OB8Wgdk6I8/s320/image009-756125.png" /></a><o:p></o:p></div>
<div class="MsoListParagraph">
<br /></div>
<div class="MsoNormal">
<b>Let see the reverse way i.e. how to propagate the changes made in BPM studio to BPM composer.<o:p></o:p></b></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoListParagraph" style="mso-list: l0 level1 lfo3; text-indent: -.25in;">
1. Add some changes to process. For demo, we will add note.<o:p></o:p></div>
<div class="MsoListParagraph">
<a href="http://4.bp.blogspot.com/-q9vS_emKdys/VtbdAYLy3XI/AAAAAAAAWJQ/GN_vVvy8r9E/s1600/image010-757370.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6257431730839543154" src="https://4.bp.blogspot.com/-q9vS_emKdys/VtbdAYLy3XI/AAAAAAAAWJQ/GN_vVvy8r9E/s320/image010-757370.png" /></a><o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l0 level1 lfo3; text-indent: -.25in;">
2. Right Click the project->Versioning->Commit. Add comments and click OK<o:p></o:p></div>
<div class="MsoListParagraph">
<a href="http://2.bp.blogspot.com/-pdnJg3MZBDA/VtbdAtxbscI/AAAAAAAAWJY/DcRI_PFUdO4/s1600/image011-758360.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6257431736634552770" src="https://2.bp.blogspot.com/-pdnJg3MZBDA/VtbdAtxbscI/AAAAAAAAWJY/DcRI_PFUdO4/s320/image011-758360.png" /></a><o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l0 level1 lfo3; text-indent: -.25in;">
3. If you face below issue:<o:p></o:p></div>
<div class="MsoListParagraph">
<a href="http://4.bp.blogspot.com/-QuaWVopWOvo/VtbdA3IMxhI/AAAAAAAAWJg/gp8257IsbMY/s1600/image012-759646.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6257431739145963026" src="https://4.bp.blogspot.com/-QuaWVopWOvo/VtbdA3IMxhI/AAAAAAAAWJg/gp8257IsbMY/s320/image012-759646.png" /></a><o:p></o:p></div>
<div class="MsoListParagraph">
“svn: E204900: Commit failed (details follow): svn: E204900: E170001: Authorization failed”<o:p></o:p></div>
<div class="MsoListParagraph">
<br /></div>
<div class="MsoListParagraph">
It occurs is the project is locked in BPM composer by some user. We need to unlock the project from BPM composer whenever we are not using it.<o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l0 level1 lfo3; text-indent: -.25in;">
4. To Unlock, go to BPM/Composer in browser. / Close the project from BPM composer.<o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l0 level1 lfo3; text-indent: -.25in;">
5. Click on Order space -> Modeler View -> Administrator View<o:p></o:p></div>
<div class="MsoListParagraph">
<a href="http://2.bp.blogspot.com/-0uZER2yOhfk/VtbdBZu8jxI/AAAAAAAAWJo/DPgHZsiuVxc/s1600/image013-760874.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6257431748435283730" src="https://2.bp.blogspot.com/-0uZER2yOhfk/VtbdBZu8jxI/AAAAAAAAWJo/DPgHZsiuVxc/s320/image013-760874.png" /></a><o:p></o:p></div>
<div class="MsoListParagraph">
And unlock the BPM project.<o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l0 level1 lfo3; text-indent: -.25in;">
6. Now check in the BPM JDEV project.<o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l0 level1 lfo3; text-indent: -.25in;">
8. In order to verify the changes. Login to BPM composer, go to the process that we created, you will be able to see the note added in BPM studio.<o:p></o:p></div>
<div class="MsoListParagraph">
<a href="http://3.bp.blogspot.com/-PcZ59ZlxD4M/VtbdBtTEJuI/AAAAAAAAWJw/Eec3GuPYdY0/s1600/image014-762112.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6257431753687049954" src="https://3.bp.blogspot.com/-PcZ59ZlxD4M/VtbdBtTEJuI/AAAAAAAAWJw/Eec3GuPYdY0/s320/image014-762112.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
In this way we can use PAM feature for smooth collaboration.<o:p></o:p></div>
<div class="MsoNormal">
Thanks.<o:p></o:p></div>
</div>
<br /></div>
Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-6794117108223651332016-02-28T15:26:00.006+05:302016-03-03T00:32:35.499+05:30JAVA embedding in Oracle SOA 12c<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="WordSection1">
<div class="MsoNormal">
In this post we will explore how to EMBED Java or call Java API from Oracle SOA BPEL process.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
In order to demonstrated this, I have created a simple JAVA project with 3 classes.<br />
<span style="text-indent: -0.25in;">1.<span style="font-family: "times new roman"; font-size: 7pt; font-stretch: normal;"> </span></span><span style="text-indent: -0.25in;">Account.java -> This class represent Account DTO</span></div>
<div class="MsoListParagraph" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<span style="mso-list: Ignore;"> 2.<span style="font: 7.0pt "Times New Roman";"> </span></span>Party.java -> This class represent a party (customer) DTO<o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<span style="mso-list: Ignore;"> 3.<span style="font: 7.0pt "Times New Roman";"> </span></span>GetNumber.java -> This class contains the method that takes Party DTO as input and apply some business logic and return party’s AccountDTO in response.<o:p></o:p></div>
<div class="MsoListParagraph">
public Account getNumber(Party party)<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
The above classes are created to simulate the calling of actual business logic API’s.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<table border="1" cellpadding="0" cellspacing="0" class="MsoTableGrid" style="border-collapse: collapse; border: none;"> <tbody>
<tr> <td style="border: solid 1.0pt; padding: 0in 5.4pt 0in 5.4pt; width: 155.8pt;" valign="top" width="208"><div class="MsoNormal">
Account.java<o:p></o:p></div>
</td> <td style="border-left: none; border: solid 1.0pt; padding: 0in 5.4pt 0in 5.4pt; width: 155.85pt;" valign="top" width="208"><div class="MsoNormal">
Party.java<o:p></o:p></div>
</td> <td style="border-left: none; border: solid 1.0pt; padding: 0in 5.4pt 0in 5.4pt; width: 254.95pt;" valign="top" width="340"><div class="MsoNormal">
GetNumber.java<o:p></o:p></div>
</td> </tr>
<tr> <td style="border-top: none; border: solid 1.0pt; padding: 0in 5.4pt 0in 5.4pt; width: 155.8pt;" valign="top" width="208"><div class="MsoNormal">
public class Account {<o:p></o:p></div>
<div class="MsoNormal">
public Account() {<o:p></o:p></div>
<div class="MsoNormal">
super();<o:p></o:p></div>
<div class="MsoNormal">
}<o:p></o:p></div>
<div class="MsoNormal">
String account;<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
public void setAccount(String account) {<o:p></o:p></div>
<div class="MsoNormal">
this.account = account;<o:p></o:p></div>
<div class="MsoNormal">
}<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
public String getAccount() {<o:p></o:p></div>
<div class="MsoNormal">
return account;<o:p></o:p></div>
<div class="MsoNormal">
}<o:p></o:p></div>
<div class="MsoNormal">
}<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
</td> <td style="border-bottom: solid 1.0pt; border-left: none; border-right: solid 1.0pt; border-top: none; padding: 0in 5.4pt 0in 5.4pt; width: 155.85pt;" valign="top" width="208"><div class="MsoNormal">
public class Party {<o:p></o:p></div>
<div class="MsoNormal">
public Party() {<o:p></o:p></div>
<div class="MsoNormal">
super();<o:p></o:p></div>
<div class="MsoNormal">
}<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
public void setName(String name) {<o:p></o:p></div>
<div class="MsoNormal">
this.name = name;<o:p></o:p></div>
<div class="MsoNormal">
}<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
public String getName() {<o:p></o:p></div>
<div class="MsoNormal">
return name;<o:p></o:p></div>
<div class="MsoNormal">
}<o:p></o:p></div>
<div class="MsoNormal">
String name;<o:p></o:p></div>
<div class="MsoNormal">
}<o:p></o:p></div>
</td> <td style="border-bottom: solid 1.0pt; border-left: none; border-right: solid 1.0pt; border-top: none; padding: 0in 5.4pt 0in 5.4pt; width: 254.95pt;" valign="top" width="340"><div class="MsoNormal">
public class GetNumber {<o:p></o:p></div>
<div class="MsoNormal">
public GetNumber() {<o:p></o:p></div>
<div class="MsoNormal">
super();<o:p></o:p></div>
<div class="MsoNormal">
}<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
public Account getNumber(Party party){<o:p></o:p></div>
<div class="MsoNormal">
Account account=new Account();<o:p></o:p></div>
<div class="MsoNormal">
//Just copying input to output for demo<o:p></o:p></div>
<div class="MsoNormal">
account.setAccount("Hello "+party.getName());<o:p></o:p></div>
<div class="MsoNormal">
return account;<o:p></o:p></div>
<div class="MsoNormal">
}<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
}<o:p></o:p></div>
</td> </tr>
</tbody> </table>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Now we need to create the JAR of the project. <o:p></o:p></div>
<div class="MsoNormal">
Right the Java project->deploy as a jar<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<a href="http://2.bp.blogspot.com/-skFY6eAAV8k/VtAnazeMR3I/AAAAAAAAWBc/Jj2JIGWkqZ8/s1600/image001-754929.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6255543223864543090" src="https://2.bp.blogspot.com/-skFY6eAAV8k/VtAnazeMR3I/AAAAAAAAWBc/Jj2JIGWkqZ8/s320/image001-754929.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Now let’s create New SOA project (JAVASOAEMBED). This will create a new project in JDEV.<o:p></o:p></div>
<div class="MsoNormal">
<a href="http://4.bp.blogspot.com/-sY_6UY6bFL4/VtAnbED9CII/AAAAAAAAWBk/rckbUgjkpec/s1600/image002-756349.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6255543228317894786" src="https://4.bp.blogspot.com/-sY_6UY6bFL4/VtAnbED9CII/AAAAAAAAWBk/rckbUgjkpec/s320/image002-756349.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Now create new sync BPEL 2.0 process in composite.xml. <o:p> </o:p></div>
<div class="MsoNormal">
Request: string input<o:p></o:p></div>
<div class="MsoNormal">
Response string output<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<a href="http://1.bp.blogspot.com/-3Yxnbt_J2YA/VtAnbTSJnGI/AAAAAAAAWBs/_C9xow2WPy0/s1600/image003-757575.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6255543232403971170" src="https://1.bp.blogspot.com/-3Yxnbt_J2YA/VtAnbTSJnGI/AAAAAAAAWBs/_C9xow2WPy0/s320/image003-757575.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Now add the JAVA Embedding component from oracle extensions to BPEL Process.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Name it as : call Java<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<a href="http://1.bp.blogspot.com/-7zlWB7ksaOY/VtAnbjmAqCI/AAAAAAAAWB0/ADPk9qp2c3c/s1600/image004-758764.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6255543236782237730" src="https://1.bp.blogspot.com/-7zlWB7ksaOY/VtAnbjmAqCI/AAAAAAAAWB0/ADPk9qp2c3c/s320/image004-758764.png" /></a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Now we will add the code to JAVA activity to call the dummy business API the we created above.<o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l2 level1 lfo2; text-indent: -.25in;">
<span style="mso-list: Ignore;"> 1.<span style="font: 7.0pt "Times New Roman";"> </span></span>Add the JAR as a library reference to your JDEV so that it does not give compilation error.<o:p></o:p></div>
<div class="MsoListParagraph">
Right Click -> Project Properties -> Libraries and classpath-> add the JAR<o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l2 level1 lfo2; text-indent: -.25in;">
<span style="mso-list: Ignore;"> 2.<span style="font: 7.0pt "Times New Roman";"> </span></span>In BPEL 2.0 all the imports for the external classes need to be specific in BPEL process source code.<o:p></o:p></div>
<div class="MsoListParagraph">
I.e.<o:p></o:p></div>
<div class="MsoListParagraph">
Click on Source code of the BPEL process. Under Process element add below imports<o:p></o:p></div>
<div class="MsoListParagraph">
<br /></div>
<div class="MsoListParagraph">
<import location="oracle.xml.parser.v2.XMLElement" importType="http://schemas.oracle.com/bpel/extension/java"/><o:p></o:p></div>
<div class="MsoListParagraph">
<import location="testjavasoa.Account" importType="http://schemas.oracle.com/bpel/extension/java"/><o:p></o:p></div>
<div class="MsoListParagraph">
<import location="testjavasoa.Party" importType="http://schemas.oracle.com/bpel/extension/java"/><o:p></o:p></div>
<div class="MsoListParagraph">
<import location="testjavasoa.GetNumber" importType="http://schemas.oracle.com/bpel/extension/java"/><o:p></o:p></div>
<div class="MsoListParagraph">
<br /></div>
<div class="MsoListParagraph">
<br /></div>
<div class="MsoListParagraph" style="mso-list: l2 level1 lfo2; text-indent: -.25in;">
3. Have added below code which simply takes the input from payload and pass it to JAVA API. Also copy the output from JAVA API to response variable.<o:p></o:p></div>
<div class="MsoNormal">
Please find the comments below to under the code in more details.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
try { <o:p></o:p></div>
<div class="MsoNormal">
<o:p></o:p></div>
<div class="MsoNormal">
//Fetch the element value from InputVariable using getVariableData() function.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
XMLElement input <o:p></o:p></div>
<div class="MsoNormal">
= (XMLElement) getVariableData("inputVariable", "payload", "/client:process/client:input"); <o:p></o:p></div>
<div class="MsoNormal">
addAuditTrailEntry("Input "+ input.getTextContent()); <o:p> </o:p></div>
<div class="MsoNormal">
String name = input.getTextContent(); <o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
//Pass the input payload data to JAVA API <o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
GetNumber getNumber = new GetNumber(); <o:p></o:p></div>
<div class="MsoNormal">
Party p=new Party(); <o:p></o:p></div>
<div class="MsoNormal">
p.setName(name); <o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
//The function allow us to add the audit data which is visible when we open the instance flow from EM console.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
addAuditTrailEntry(getNumber.getNumber(p).getAccount()); <o:p> </o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
//the function will set the data in outputVarialble <o:p></o:p></div>
<div class="MsoNormal">
setVariableData("outputVariable", "payload", "/client:processResponse/client:result", getNumber.getNumber(p).getAccount()); <o:p></o:p></div>
<div class="MsoNormal">
<o:p></o:p></div>
<div class="MsoNormal">
} catch (Exception e) { <o:p></o:p></div>
<div class="MsoNormal">
System.out.println("exception "+e.getMessage()); <o:p></o:p></div>
<div class="MsoNormal">
addAuditTrailEntry(e); <o:p></o:p></div>
<div class="MsoNormal">
}<o:p></o:p></div>
<div class="MsoNormal">
<o:p></o:p></div>
<div class="MsoNormal">
For more info on available function please reference below link:<br />
<a href="http://docs.oracle.com/cd/E15586_01/integration.1111/e10224/bp_java.htm"><span style="color: windowtext; text-decoration: none;">http://docs.oracle.com/cd/E15586_01/integration.1111/e10224/bp_java.htm</span></a><o:p></o:p></div>
<div class="MsoNormal">
Table 13-1 Built in Methods for bpelx:exec<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Please Note below points:<o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l1 level1 lfo3; text-indent: -.25in;">
1. setVariableData function requires the variable to be initialized prior. Thus if you try to assign value without initializing then it will thrown Invalid or empty variable exception. For this I added the assign activity before JAVA callout in order to initialize the outputVariable with a blank string.<o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l1 level1 lfo3; text-indent: -.25in;">
2. Before deploying the SOA project you need to configure the BPEL classpath properties by mentioning the required JARs.<o:p></o:p></div>
<div class="MsoListParagraph">
<a href="http://1.bp.blogspot.com/-qEeyItH0URg/VtAncG8a5sI/AAAAAAAAWB8/mvcWZaqPivo/s1600/image005-759999.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6255543246271473346" src="https://1.bp.blogspot.com/-qEeyItH0URg/VtAncG8a5sI/AAAAAAAAWB8/mvcWZaqPivo/s320/image005-759999.png" /></a><o:p></o:p></div>
<div class="MsoListParagraph" style="mso-list: l1 level1 lfo3; text-indent: -.25in;">
3. If you face the below “NoClassDefFoundError”, you need to put the jar file in SCA-INF/Lib folder and redploy. <o:p></o:p></div>
<div class="MsoListParagraph">
####<Feb 25, 2016 7:07:48 PM IST> <Error> <oracle.soa.bpel.system> <SML-0646> <DefaultServer> <[ACTIVE] ExecuteThread: '6' for queue: 'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <BEA1-5236741793AD9E472B1E> <4bbddfd6-7fb6-40e9-9ede-6da1762b9bea-000005dd> <1456407468639> <BEA-000000> <Unhandled BPELFault:faultName: {{http://schemas.oracle.com/bpel/extension}runtimeFault}<o:p></o:p></div>
<div class="MsoListParagraph">
messageType: {{http://schemas.oracle.com/bpel/extension}RuntimeFaultMessage}<o:p></o:p></div>
<div class="MsoListParagraph">
parts: {{<o:p></o:p></div>
<div class="MsoListParagraph">
summary=<summary>testjavasoa/GetNumber</summary><o:p></o:p></div>
<div class="MsoListParagraph">
,detail=<detail>java.lang.NoClassDefFoundError: testjavasoa/GetNumber<o:p></o:p></div>
<div class="MsoListParagraph">
at orabpel.bpelprocess1.ExecLetBxExe0.execute(ExecLetBxExe0.java:69)<o:p></o:p></div>
<div class="MsoListParagraph">
at com.collaxa.cube.engine.ext.bpel.common.wmp.BPELxExecWMP.__executeStatements(BPELxExecWMP.java:42)<o:p></o:p></div>
<div class="MsoListParagraph">
at com.collaxa.cube.engine.ext.bpel.common.wmp.BaseBPELActivityWMP.perform(BaseBPELActivityWMP.java:188)<o:p></o:p></div>
<div class="MsoListParagraph">
at com.collaxa.cube.engine.CubeEngine.performActivity(CubeEngine.java:2864)<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
In this way we can call JAVA API from SOA BPEL process.<o:p></o:p></div>
</div>
<br /></div>
Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-28166965249343805492016-02-12T09:10:00.003+05:302016-02-12T09:10:55.001+05:30configuring the http proxy in weblogic<div dir="ltr" style="text-align: left;" trbidi="on">
To configure the proxy in weblogic configure the below properties in setDomainEnv:<br />
<br />
-DproxySet=true<br />
-Dweblogic.webservice.transport.http.proxy.host=proxy<br />
-Dweblogic.webservice.transport.http.proxy.port=80<br />
-Dweblogic.webservice.transport.https.proxy.host=proxy<br />
-Dweblogic.webservice.transport.https.proxy.port=80<br />
-Dhttp.proxyHost=proxy<br />
-Dhttp.proxyPort=80<br />
-Dhttp.nonProxyHosts="172.16.40.84|172.16.40.63"<br />
-Dhttps.proxyHost=proxy<br />
-Dhttps.proxyPort=80<br />
-Dhttps.nonProxyHosts="172.16.40.84|172.16.40.63"<br />
<br />
If the proxy is accepting username/password<br />
Or if you face the below issue,<br />
weblogic.net.http.HttpUnauthorizedException: Auth scheme Negotiate is not supported!<br />
then you may need to write the custom authenticator for calling the service behind proxy.<br />
<br />
Please find the below example for custom proxy authenticator:<br />
<br />
Write the class that extends java.net.Authenticator.<br />
***<br />
import java.net.Authenticator;<br />
import java.net.PasswordAuthentication;<br />
<br />
public class ProxyAuthenticator extends Authenticator {<br />
private String username;<br />
private String password;<br />
public ProxyAuthenticator(String username, String password){<br />
this.username = username;<br />
this.password = password;<br />
}<br />
public PasswordAuthentication getPasswordAuthentication () {<br />
return new PasswordAuthentication (username, password.toCharArray());<br />
}<br />
<br />
}<br />
***<br />
<br />
From your code, before calling the service over the http set the above authenticator.<br />
<br />
//I set the proxy again for safer side<br />
System.setProperty("http.proxyHost", "proxy"); <br />
System.setProperty("http.proxyPort", "80");<br />
System.setProperty("https.proxyHost", "proxy"); <br />
System.setProperty("https.proxyPort", "80");<br />
<br />
Authenticator.setDefault (new ProxyAuthenticator("username","password"));<br />
<br />
<br />
Reference Documents:<br />
<br />
https://docs.oracle.com/cd/E13222_01/wls/docs81/webserv/client.html (Using a Proxy Server with the WebLogic Web Services Client)<br />
https://docs.oracle.com/cd/E24329_01/web.1211/e24967/client.htm#WSRPC215<br />
http://soadev.blogspot.com/2010/02/setting-proxy-authentication-in-java.html<br />
<div>
<br /></div>
</div>
Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-27375066583766037952015-11-13T15:01:00.001+05:302015-11-13T15:01:12.066+05:30Configuring Human Task Webform URL in BPM/SOA<div dir="ltr" style="text-align: left;" trbidi="on">
For every human task, the webform URI i.e. Hostname and Port can be configure using below two methods.<div>
1. Manually changing or correcting the URI in EM console.</div>
<div>
2. Specifying the Hostname and Port in hwtaskflow.xml of UI project before deployment.</div>
<div>
<br /></div>
<div>
But, many times, this configuration is missed during deployment time. </div>
<div>
In some cases, we there is a cluster environment with HTTP server configured over SSL. In this case, we do not want the user to use the HTTP URL, or manually change the protocol and Port to Http.</div>
<div>
<br /></div>
<div>
In order to handle the above situation, we can configure the URL permanently in SOA workflow properties. </div>
<div>
SOA Administration->WorkFlow Properties-> Look for <b>FusionAppsFrontEndHostUrl </b>property.</div>
<div>
<br /></div>
<div>
Configure this property to the hostname and port which we want the webform to always render on.</div>
<div>
<br /></div>
<div>
Once this property is configured, even if we missed to configure human task URI or configure the hostname incorrectly. The server will redirect the request to the URL configured in <b>FusionAppsFrontEndHostUrl property.</b></div>
<div>
<b><br /></b></div>
<div>
Please find the below screenshot for more reference.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-HaGDpwCWktk/VkWt0H_M9UI/AAAAAAAAPVQ/nprda5E6Owo/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="177" src="http://2.bp.blogspot.com/-HaGDpwCWktk/VkWt0H_M9UI/AAAAAAAAPVQ/nprda5E6Owo/s320/Capture.JPG" width="320" /></a></div>
<div>
<br /></div>
</div>
Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-50634702337741674842015-11-07T01:07:00.002+05:302015-11-07T01:07:45.330+05:30Installing Rasbian on Raspberry PI 2 without Monitor<div dir="ltr" style="text-align: left;" trbidi="on">
This post will explain the step by step process to install Rasbain OS on Raspberry PI 2 without connecting to monitor or keyboard using windows machine.<br />
<br />
I ordered Raspberry PI 2 and was quiet excited to get started. I only had the Raspberry PI and 8GB SD card. I have a monitor but it does not have HDMI port. I also did not have the HDMI to VGA convertor, thus was not able to connect to monitor.<br />
I had my laptop, so based on the various post online I installed Rasbain on my PI by following below steps.<br />
<br />
<b>1</b>. Download the RASBIAN OS from Raspberry PI site. I downloaded <b>"RASPBIAN JESSIE 4.1".</b><br />
<br />
https://www.raspberrypi.org/downloads/raspbian/<br />
<br />
The Rasbian ZIP, "2015-09-24-raspbian-jessie" contain <b>2015-09-24-raspbian-jessie.iso</b> file.<br />
<br />
<b>2</b>. Download SD Card Formatter. I recommend not to format the SD card using windows format feature. Use SD Card Formatter to format the SD.<br />
<br />
https://www.sdcard.org/downloads/formatter_4/<br />
<br />
You will need Micro SD card and a Card Reader.<br />
Note: Be careful while formatting as you may end you formatting wrong drive.<br />
<br />
<b>3. </b>Writing Rasbian Image to SD Card.<br />
We now need to write the RASBIAN OS on to SD card. Since its an ISO file we will need an ISO reader.<br />
Download and Install "win 32 disk imager".<br />
http://sourceforge.net/projects/win32diskimager/<br />
<br />
Run win32diskimager, Select the "2015-09-24-raspbian-jessie", Select the SD card and Click Write.<br />
It will take around 5mins to copy the image on to SD card.<br />
<br />
Once completed, now SD card is installed with Rasbian OS.<br />
<br />
i. Insert the SD card in to PI<br />
ii. Connect the PI to Router. I used my old LAN cable. This is required as we will be connecting to PI over SSH.<br />
iii. Power on the supply and wait for 1-2mins.<br />
<br />
<b>Note: For power supply, I used my samsung 5v charger</b><br />
<b><br /></b>
<b>4. </b>Now we need to identify PI IP address. In order to find the PI IP address, download "Advance IP Scanner"<br />
http://www.advanced-ip-scanner.com/<br />
<br />
This will list all the devices connected over a LAN. You will see Raspberry PI Foundation IP address.<br />
<br />
<b>5. </b>Download Putty client.<br />
www.putty.org<br />
i. Enter PI ip address in putty and select SSH and click connect.<br />
<br />
ii. It will prompt for the Username/Password.<br />
Rasbian default username password is <b>pi/raspberrypi</b><br />
<br />
Till now we have successfully install Rasbian on Raspberry PI 2.<br />
<br />
If you need the Visual Control i.e. Desktop, you can install VNC server on PI and VNC client on local machine.<br />
I followed below link to configure VNC.<br />
<br />
VNC:<br />
https://www.raspberrypi.org/documentation/remote-access/vnc/<br />
https://www.raspberrypi.org/documentation/remote-access/vnc/windows.md<br />
<br />
Please comment if you need further details. Will add the images for more details.</div>
Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-26139024055913350882015-10-28T18:08:00.001+05:302015-10-28T18:08:19.094+05:30Failed to instantiate workflow rule repository service (Oracle BPM 12c)<div dir="ltr" style="text-align: left;" trbidi="on">
<BEA-000000> <<oracle.bpel.services.common.ServicesLogger> <ServicesLogger> <__logException> <<.> exception.code:30704<br />
exception.type: ERROR<br />
exception.severity: 2<br />
exception.name: Failed to instantiate workflow rule repository service.<br />
exception.description: Could not create repository service for workflow rule dictionary oracle.bpel.services.workflow.user, package {2}.<br />
exception.fix: Check underlying exception. Ensure dictionary name and package are valid.<br />
><br />
ORABPEL-30704<br />
<br />
exception.code:30704<br />
exception.type: ERROR<br />
exception.severity: 2<br />
exception.name: Failed to instantiate workflow rule repository service.<br />
exception.description: Could not create repository service for workflow rule dictionary oracle.bpel.services.workflow.user, package {2}.<br />
exception.fix: Check underlying exception. Ensure dictionary name and package are valid.<br />
<div>
<br /></div>
<div>
If You put the server in trace, you will see below error:</div>
<div>
<div>
####<Oct 28, 2015 5:51:54 PM IST> <Notice> <Stdout> <SML-0646> <DefaultServer> <[ACTIVE] ExecuteThread: '12' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <d6df2caf-b25e-4d39-8228-2161a8e2a65f-00004a92> <1446034914888> <BEA-000000> <<oracle.bpel.services.common.ServicesLogger> <ServicesLogger> <__log> <<OracleRuleRepositoryService.initDMDictionary> Initializing data model dictionary failed with error: RUL-05401: The path "file:/C:/Oracle12c/Middleware/Oracle_Home/soa/soa/modules/oracle.soa.workflow_11.1.1/bpm-services.jar" does not exist. Ensure that the path exists on the host system.</div>
</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
I had the installation in D:\Oracle12c but the server was checking in C:\Oracle12c.</div>
<div>
<br /></div>
<div>
The Quick fix for this issue is create a similar directory structure with the file on C: Drive.</div>
</div>
Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-17320586662221672642015-10-13T17:51:00.000+05:302015-10-13T17:51:04.188+05:30Unknown Protocol: servicebus error in Oracle SOA 12.1.3<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<br />
I started getting this error while deploying simple OSB project on integrated SOA server. Initially i thought its an error with my WSDL file, but in actual it a bug. Based on the posts online, applied the below two solution which worked for me.<br />
<br />
<b>Error:</b><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-Fxm7a5sh4G8/Vhz3G8_a0ZI/AAAAAAAAO98/bC9JybB1aXQ/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="118" src="http://2.bp.blogspot.com/-Fxm7a5sh4G8/Vhz3G8_a0ZI/AAAAAAAAO98/bC9JybB1aXQ/s320/Capture.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<b>Solution 1:</b><br />
<br />
1. Login to weblogic console.<br />
2. Click on Services ->OSGi Frameworks<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/--od3a8Tbs-E/Vhz1f_HWlPI/AAAAAAAAO9k/L-gJ8iOaIOQ/s1600/Capture1.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="200" src="http://4.bp.blogspot.com/--od3a8Tbs-E/Vhz1f_HWlPI/AAAAAAAAO9k/L-gJ8iOaIOQ/s200/Capture1.JPG" width="166" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
3. Click on "bac-svnserver-osgi-framework"<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-s3PXbeZb7Jk/Vhz11bYEMUI/AAAAAAAAO9s/xZ-wCn1Xquc/s1600/Capture1.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="114" src="http://4.bp.blogspot.com/-s3PXbeZb7Jk/Vhz11bYEMUI/AAAAAAAAO9s/xZ-wCn1Xquc/s320/Capture1.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
4. Click on targets<br />
5. Remove Default server from the target.<br />
6. Click on SAVE.<br />
<br />
<b>Solution 2:</b><br />
<br />
1. Click on "bac-svnserver-osgi-framework"<br />
2. Click on configuration<br />
3. Under Init properties insert this property "felix.service.urlhandlers=false"<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-Lx57SGWdyqo/Vhz2wiysKEI/AAAAAAAAO90/GKoG5MIxnLo/s1600/Capture1.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="133" src="http://1.bp.blogspot.com/-Lx57SGWdyqo/Vhz2wiysKEI/AAAAAAAAO90/GKoG5MIxnLo/s320/Capture1.JPG" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Post this, <b>Restart the Integrated Server.</b><br />
Thanks<br />
<b><br /></b>
<br />
<br /></div>
Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.comtag:blogger.com,1999:blog-5549352010919862857.post-29038336738700568652015-09-10T16:52:00.002+05:302015-09-10T16:58:38.058+05:30Generating PDF Report using Apache FOP, MS word (for template) and JAVA<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="line-height: 1.295; margin-bottom: 8pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Generating PDF Report using Apache FOP, MS word (for template) and JAVA</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 8pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">We often encounter the requirement for generating the custom PDF report. There are many open source tools like iText, Jasper etc. I was looking for some approach that ease the formatting task for the report. </span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 8pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Will follow below steps to generate the report:</span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-YmMFAKuYHg8/VfFpV5TzARI/AAAAAAAAOcg/G4hyXoTK3Xg/s1600/Capture.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="35" src="http://3.bp.blogspot.com/-YmMFAKuYHg8/VfFpV5TzARI/AAAAAAAAOcg/G4hyXoTK3Xg/s400/Capture.JPG" width="400" /></a></div>
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 8pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 8pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">For further understanding, we will take a task to generate the sample report. We will generate user bank account statement PDF. As this is the sample PDF, we will include limited details. The logo is </span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 8pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">We will follow the above mentioned steps:</span></div>
<ol style="margin-bottom: 0pt; margin-top: 0pt;">
<li dir="ltr" style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; list-style-type: decimal; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Prepare the report template using MS word.</span></div>
</li>
</ol>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">As a part of the bank statement report we will need to include below information in report.</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 12px; font-style: italic; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Name</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 12px; font-style: italic; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Address</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 12px; font-style: italic; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Account Number</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 12px; font-style: italic; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Bank Name</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 12px; font-style: italic; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Month</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 12px; font-style: italic; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Transaction List-> Desc, Credit/Debit, Amount</span></div>
<b id="docs-internal-guid-a99fbc3f-b6f4-6f48-a1ab-33e6a05d4e6d" style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Based on the above mentioned parameter we generated the below PDF template in MS word. Generating this template in MS word and formatting it does not take long time. If you observe, we have mentioned the parameter with #VALUE. This will help us replace the #value with correct XPATH.</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img height="216" src="https://lh5.googleusercontent.com/2vU-7AIfpNAw_u5Q6gMtaN1pII8HaSMlM-NtoOty-SgqiHi4PPxZXv-R1RlP1gHheQn8uVrjn8Jiv5-6Tg0zEb7XOZOXHNyJwY-K3B_gkL3KvHbM6rFfjLAEWfsQbVZ1WbYGBm2Qzuc9XvDI" style="border: none; transform: rotate(0rad);" width="400" /></span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<ol start="2" style="margin-bottom: 0pt; margin-top: 0pt;">
<li dir="ltr" style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; list-style-type: decimal; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Export or Save MS Word Template as XML-FO</span></div>
</li>
</ol>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">In order to export the MS word as XML-FO we need to download and run “OfficeWordWordMLtoXSL-FO.exe”.</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">You can download the exe from </span><a href="https://msdn.microsoft.com/en-us/library/aa537167(v=office.11).aspx" style="text-decoration: none;"><span style="background-color: transparent; color: #0563c1; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">https://msdn.microsoft.com/en-us/library/aa537167(v=office.11).aspx</span></a><span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">.</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Run the exe, this will generate the folder with .XSL files</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Open the word template, </span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Click on File,</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Save As,</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Select word 2003 XML Document,</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Check Apply Transform,</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Select “Word2FO.xsl” file under the folder “OfficeWordWordMLtoXSL-FO”</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Click Save.</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img height="327" src="https://lh4.googleusercontent.com/5H0nH_iYzjO_3EMoAUvvfNd0THtZx8OQawmnc6BehTbxVhOhVbqoTqbe7yv1w5ewMsI2ektof4yNO8V_T1Xnv9cN7Re2xMzKGeR_P4EifRFNJ2_a9NcdpFbjhJIMrQhIyPBbifxBdwN1pw-g" style="border: none; transform: rotate(0rad);" width="400" /></span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">This will generate the StatementTemplate.xml file in the corresponding folder.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Next step is to convert the word XML to XSLT,</span></div>
<ul style="margin-bottom: 0pt; margin-top: 0pt;">
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Create a new XSLT template file.</span></div>
</li>
</ul>
<blockquote class="tr_bq" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 72pt; margin-top: 0pt; text-align: left;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"><?xml version="1.0" encoding="utf-8"?></span></span><span style="background-color: transparent; color: black; font-family: Calibri; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"><xsl:stylesheet version="1.0"</span></span><span style="font-size: x-small;"><span style="background-color: transparent; color: black; font-family: Calibri; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span><span style="background-color: transparent; color: black; font-family: Calibri; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">xmlns:lc="http://www.abc.in/bank"</span></span><span style="font-size: x-small;"><span style="background-color: transparent; color: black; font-family: Calibri; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span><span style="background-color: transparent; color: black; font-family: Calibri; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:fo="http://www.w3.org/1999/XSL/Format"></span></span><span style="font-size: x-small;"><span style="background-color: transparent; color: black; font-family: Calibri; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span><span style="background-color: transparent; color: black; font-family: Calibri; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><xsl:output method="xml" indent="yes" /></span></span><span style="font-size: x-small;"><span style="background-color: transparent; color: black; font-family: Calibri; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span><span style="background-color: transparent; color: black; font-family: Calibri; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><xsl:template match="/"></span></span><span style="font-size: x-small;"><span style="background-color: transparent; color: black; font-family: Calibri; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span><span style="background-color: transparent; color: black; font-family: Calibri; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span></span><span style="font-size: x-small;"><span style="background-color: transparent; color: black; font-family: Calibri; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span><span style="background-color: transparent; color: black; font-family: Calibri; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"></xsl:template></span></span><span style="background-color: transparent; color: black; font-family: Calibri; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"></xsl:stylesheet></span></span></blockquote>
<ul style="margin-bottom: 0pt; margin-top: 0pt;">
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Copy the entire “<fo:root>” element from the generated statementTemplate.xml file and paste it inside “<xsl:template>” element.</span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Save the file as StatementTemplate.xsl</span></div>
</li>
</ul>
<ol start="3" style="margin-bottom: 0pt; margin-top: 0pt;">
<li dir="ltr" style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; list-style-type: decimal; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Generate the XML for the date in report template</span></div>
</li>
</ol>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Now we will need to generate the XML for our report data. This is user specific based on user XSD. For this example, I will generate a simple XML.</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img height="323px;" src="https://lh6.googleusercontent.com/pBWZU1oxDFID7KRuL_63cDutA5MtbgqZr0mxk7s4jDeZV8SWoTXdG3hMDKjJct-enzRDO_VL_b-UUmqJOmON_blv4CAHLv5319j9A7gdqBHcKiz1g58y6PzKhcUZ13yY4VkfPeQjYzOs2Ast" style="-webkit-transform: rotate(0.00rad); border: none; transform: rotate(0.00rad);" width="284px;" /></span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<ol start="4" style="margin-bottom: 0pt; margin-top: 0pt;">
<li dir="ltr" style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; list-style-type: decimal; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Now replace the #VALUE in XSL with the XPATH based on the XML.</span></div>
</li>
</ol>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">We will use simple XSL syntax to replace the value at runtime.</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Example: </span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Replace : #NAME with </span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: red; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><xsl:value-of select="bankstatment/name" /></span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">The row of the table in FO is represented by “fo:table-row”. Here as we need to dynamically print the rows based on the number transaction of the user, we will apply XSL for-loop and repeat “fo:table-row” </span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Example: “<xsl:for-each select="bankstatment/transactionlist/transaction">”</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<ol start="5" style="margin-bottom: 0pt; margin-top: 0pt;">
<li dir="ltr" style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; list-style-type: decimal; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.295; margin-bottom: 8pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Generate the PDF using Apache FOP</span></div>
</li>
</ol>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 8pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">First, you need to download Apace FOP libraries. </span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 8pt; margin-left: 36pt; margin-top: 0pt;">
<a href="https://xmlgraphics.apache.org/fop/" style="text-decoration: none;"><span style="background-color: transparent; color: #0563c1; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">https://xmlgraphics.apache.org/fop/</span></a></div>
<ul style="margin-bottom: 0pt; margin-top: 0pt;">
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Create a new JAVA project.</span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Copy the XML and XSL file in to the java project.</span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Give all the JARs under FOP bin as a dependency to the project.</span></div>
</li>
</ul>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-left: 72pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">D:\ApacheFOB\fop-1.1-bin\fop-1.1\lib</span></div>
<ul style="margin-bottom: 0pt; margin-top: 0pt;">
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Run the JAVA program mention below by replacing the correct file paths.</span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.295; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">When you run for first time, the background color of the </span><span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">table</span><span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> in generate FO will be black, you need to manually change the color to white.</span></div>
</li>
</ul>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 8pt; margin-left: 72pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Look for the property “background-color="black"” and replace it with background-color="white"</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 8pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">If all run properly, PDF report will be generated as per the format mentioned in the WORD template. </span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 8pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Please find the generated PDF:</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 8pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><a href="https://drive.google.com/file/d/0B8n7aI4e8I_rQklEMzNuT3Fxam8/view?usp=sharing" target="_blank">Download StatementPDF</a></span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 8pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 8pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Please Note: you will need to manually edit the xsl:FO in order to adjust some alignment in the generated PDF.</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 8pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Links:</span></div>
<div dir="ltr" style="line-height: 1.295; margin-bottom: 8pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Calibri; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Apache FOP:</span></div>
<div dir="ltr" style="margin-bottom: 8pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; font-size: 14.6667px; line-height: 18.9933px; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: Calibri;"><a href="https://xmlgraphics.apache.org/fop/">https://xmlgraphics.apache.org/fop/</a></span></span></div>
<div dir="ltr" style="margin-bottom: 8pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; font-size: 14.6667px; line-height: 18.9933px; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: Calibri;">MS word to FO Transform:</span></span></div>
<div dir="ltr" style="margin-bottom: 8pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; font-size: 14.6667px; line-height: 18.9933px; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: Calibri;"><a href="https://msdn.microsoft.com/en-us/library/aa537167(v=office.11).aspx">https://msdn.microsoft.com/en-us/library/aa537167(v=office.11).aspx</a></span></span></div>
<div dir="ltr" style="margin-bottom: 8pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; font-size: 14.6667px; line-height: 18.9933px; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="margin-bottom: 8pt; margin-left: 36pt; margin-top: 0pt;">
<span style="font-size: 14.6667px; line-height: 18.9933px; white-space: pre-wrap;">You can download the sample Application from below link:</span></div>
<div dir="ltr" style="margin-bottom: 8pt; margin-left: 36pt; margin-top: 0pt;">
<span style="font-size: 14.6667px; line-height: 18.9933px; white-space: pre-wrap;"><a href="https://drive.google.com/file/d/0B8n7aI4e8I_rZ3ZRcVFGSHY0LUU/view?usp=sharing" target="_blank">PDF Report Generation</a></span></div>
<br /></div>
Rohan Lopeshttp://www.blogger.com/profile/08368649890442262923noreply@blogger.com