#DOAG2017 wrap up

doag_wegweiser

Another DOAG is over and here are my summary.

Day 1

As my presentation is planned for noon I arrived this year on the evening before, so I could attend the early sessions on 8:30.

Sten Vesterli: APEX, ADF, or ABCS? A real-life application built in 3 tools

He compares the development of a real application in APEX, ADF and ABCS. He don’t believe that ABCS is usable from end users and shows interesting problems with the date pickers in APEX and ADF.

Welcome by DOAG and Oracle

Stefan Kinnen from DOAG welcomes the attendees and gives an short overview for the conference. The part of the Kenneth Johansen from Oracle was very dry in the contrast.

Keynote: Neil SholayTracking the Inevitable (How to plan for likely and disruptive business scenarios.)

Quite the reverse the colleague of Kenneth Johansen. He present interesting views about the ongoing disruption in every business and how to prepare for this. Unfortunately the presentation is not available for download.

Markus Lohn: Road to the (Oracle) Cloud

The next title was pure understatement. As I know Markus expanded his talk to the details of a complex SOA & ESB migration report, which most of the attendees sure does not expect. I think at least my boss not :-). I left this a little earlier as now I had to go to stage.

Torsten Kleiber: PL/SQL: Therefore test automation for those who bind themselves forever!

After several presentations about ADF and the related development life cycle I decided this year to speak about the selection process of a PL/SQL testing tool. Additional I promised to do some of this last year. The room was packed with round about 100 people, which I unfortunately never reached with ADF in the past. Despite some technical problems when starting and in the demo’s all went fine and I will thank again all my attendees. You find my presentation additional here:

Because of some discussion with my attendees and lunch I skipped the next slot.

Sabine Heimsath, Jan Karremans: #DBADev – Like dog and cat? It doesn’t have to be!

This was a “lightweight” simulated battle between DBA and > as DB developers. I little bit oversimplified, but ok, unfortunately I seen none of our DBA’s in the audience.

Alexander Weber: Cloud service – heaven and hell (application and data away)

This was report> from a real world oracle database cloud customer with an APEX app. Maybe I will remind me to look at his experience, when we will ever go the cloud.

Bryn Llewellyn: Ten Rules for Doing a PL/SQL Performance Experiment

Bryn shows how to best analyze and broke down a performance problem and the idea’s are often not restricted to PL/SQL. Lession learned “Don’t use SET TIMING ON”, as this includes additional network roundtrips.

Elmar Juergens: The sow does not get fat from weighing: PL/SQL quality analysis

I see already a similar presentation from Elmar Juergens at Javaland 2017. Same message this time. Additional he recommend to measure at first only the changed source code via a diffing algorithm to only identifying the changed lines. Most tools can’t this. Will have a look at his own tool sometime. Additional our bosses want Elmar Juergens for a presentation for our developers. Will look if this will change their minds an they will look a least a bit more at the existing measured violation.

Day 2

Markus Klenke: Better, more beautiful, more modern: barely used beads in ADF

Markus likes ADF and show this. Fine that he is at our company site to help us with this.

Daniel Hillinger, Tobias Deml: Battle: Database Virtualization

First at the beginning the decision was made, who of the both is pro and who is contra. Nice format to find the arguments for this. I’ve learned that all databases in the same virtualization farm has to be the same license and options, if you want not to pay to much. Additional it’s not usual to begin if you have fewer than 100 database instances.

Thorsten Wussow: Let the orchestra play –- Docker orchestration and visualization

Unfortunately, my expectations were disappointed here because most demos didn’t work. Not a good start to use the technology.

Dominic Weiser: Continuous integration in database development

This was another nice database CI presentation which contains utPLSQL, and the presenter had enough time to show the CI part on Jenkins too. Similar you can find in my upload too, but I had enough time in my talk to present.

Duncan Mills: Web Components, the Oracle Way

Duncan describes how to define and use own components in Oracle Jet as professional as ever. Unfortunatly I see that he has moved from ADF product management to JET architecture team now.

Michael Schulze: Tools for diagnosing Weblogic (FMW) performance problems 

A surprice for me – lot of new tools and information for my toolchain.

Markus Lohn: Quo Vadis Agile Software Development

Again Markus on the stage. He shows his Agile project experiences on several customers and I’m not sure  if he mentioned us or not.

Bryn Llewellyn: Guarding your Data Behind a Hard Shell PL/SQL API – the Detail

Bryn shows here a security model for how accessing the data via an secure API which contains of 4 schema layers: data, code, implementation, API. Unfortunatly this is not really good handled by some of Oracle own development frameworks like ADF despite you develop only on the rest API on top. But definitely give it a try.

Roland Dürre, Knud Johannsen, Christian Botta: Creative communication

The paintings was good but the rest was very esoteric presentation of philosophic messages. I think I was to inspired by the vita of mr. Botta when selected this slot, this was what I had expected.

Before the evening keynote Ulrike Schwinn was awarded, congratulations! Over my career I heard a lot of presentations from her, which was impressive, none where I’m not learned something new.

Keynote – Karl-Heinz Land: Digital Darwinism: The Silent Attack on your Business Model and Brand

Good and provocative but unfortunately in my opinion also with some mistakes. The law of moore does not apply anymore, and as this was his main explanation, why all will be automated in the near future. Yes automation will happen, but only where a real business case is. If you see that over 50% of internet traffic is eaten by video streaming already, than I think this will hamper this process. Blockchain is not an oprating system, it is an distributed secure database. But it comes with cost, which actually is not paid, for example: A Bitcoin transaction consumes more electricity than a one-person household per month. And the verification of security need to many time. This has to be solved or Blockchain will only be used productive in edge case. Generally  I’m more a fan of WildDueck.

ADF Community Meeting

Unfortunately this was greatest disappointment of the whole DOAG conference for me. Once again we discussed several things

  • ADF is not visible from Oracle because not actually marketed
  • Errors in JDeveloper: the IDE for ADF and lack of support for this
  • Provide an Virtual Machine for ADF development. This is ok for trying out, but will not supported in restricted environment at customers as at my company – a bank. At least this should not be an replacement of support for JDeveloper
  • Should we open the community to other technologies like SOA, JET, ABCS an so on. My opinion: definitely no, as this is the final dead of ADF in the community.
  • We see that some of the important product managers move to JET and does not support us anymore.

In this atmosphere this year no common dinner happens after the meeting.

After that I spent the time with some colleagues at the evening party. This year I see early clearing the rooms of the NCC. I don’t know if the starting Scope alliance party was the cause, but maybe they will do this next year on the first day instead?

Day 3

Hans Eichenberger: DBaaS with APEX 5.1 Frontend and Oracle Multitenant Option 

Will repeat my tweet here: The developer dreams come true – getting an inhouse sandbox database in minutes from the DBA! Whoops – unfortunately not my company!

Adam Lukaszewski: Boost your Forms Development with GIT and Forms API Master

At the moment I define a future proof build and deployment process at my company for the loan platform. Part of this is security and mandatory review. So the presented Forms API Master tool is the perfect tool for handling the review and merge process of branches for our legacy Forms and Reports development. Will look how to implement this in Atlassian BitBucket.

Keynote – Robert Schröder: The Difference between Errors and Failures – Experiences in Aviation 

Key message are for me: Accept that failures happen, learn from failures and change your processes accordingly. Define a clear language for your business.

Marcus Hammer: Die Hürden großer ADF-Anwendungen

Not really new for us, but as we speak a lot with other colleagues from his company the solutions are mostly known for us – and for the whole ADF community in Germany. Interesting is the decision to implement batch processing in Java too, seems that it is problematic to get PL/SQL staff right now. I’m not understanding the decision to write an own framework for database deployment, but as this was not his own topic, he could not answer why Liquibase or Flyway does not fulfill their requirements.

Frank Munz: WebLogic 12c and Java Cloud Service

I missed Franks sessions last years, but often look at his slides in the web. This was a very good entry in differentiating the different Oracle Cloud Servives, which I could use for Java Develoment. Oracle Java Cloud Service is only one of it and should correctly named Oracle WebLogic Cloud Service. Great another naming dropping: Application Container Cloud should better by named as Language as a Service. Look at the slides to get the whole picture.

Alex Nuijten: Regular Expressions: Say What?

The best example that you can learn ever something new from a good presenter, despite you think you know all about a topic and you have already read the presentation before. For an example: a regular expression looks already for the longest possible match, which could be the whole string instead of the searched pattern. Interesting too the connect by regexp_count for splitting a string into rows. And look at the standard patterns in dbms_redact before you write your own, if it is not there write it in your own package for reuse.

Frank MunzMicroservices Runtimes

The second presentation of Frank I attend compares the actual available Runtimes on the market. But special interesting was the intro into micro services: are you sure you need this and do you know, what nobody tells you about latency and so on? Or do you know that accessing another micro service can need 100000 times more than accessing a functionality in the same monolith?

Conclusion

Again this year I attend colorful bouquet of presentations and again it pay’s off to look at the whole conference program and some abstracts and presentations before to minimize disappointments.

Generally I attend to few development sessions, maybe because I use other technologies or know already the presented topic.  But my feeling is that are really fewer development presentations.

Interesting is that finally many Development, Deployment, QA, TDD and CI/CD technologies used in other development languages find it’s way into classic Oracle development technologies like Oracle PL/SQL, Forms and Reports.

For the event organization I see a little degradation of service this year, for example the quality of the food or the cleaning of toilets and rooms. And a no go: Missing water for presenters in some rooms!

That’s it!

Advertisements

, , , , , , , , , , , , , , , , , , , , , , , , , , ,

Leave a comment

Provide access to #WebLogic DMS Spy Servlet for readonly users

For security reasons and to prevent the configuration drift it is recommendable to use read only for analyzing problems.

For read only access of configuration and logs WebLogic provides out of the box the group Monitors. Unfortunately you cannot access DMS Spy Servlet with this group, which is useful for analyse runtime values of the server. Only users which belongs to the Administrators group and therefore have full access can access DMS Spy Servlet and this is not configurable in WebLogic by default.

Here you can see how to add groups on linux to the DMS Spy Servlet deployment:

pushd $ORACLE_HOME/oracle_common/modules/oracle.dms
cp dms.war dms.war.`date +%y.%m.%d.%H:%M`
unzip dms.war WEB-INF/weblogic.xml
sed -i '/Monitors/d' WEB-INF/weblogic.xml
sed -i '/^         <principal-name>Administrators<\/principal-name>$/a\         <principal-name>Monitors<\/principal-name>' WEB-INF/weblogic.xml
zip dms.war WEB-INF/weblogic.xml
rm -r WEB-INF
popd

Following is the responsible snippet in weblogic.xml before:

     <security-role-assignment>
         <role-name>Admin</role-name>
         <principal-name>Administrators</principal-name>
     </security-role-assignment>

and after modification:

     <security-role-assignment>
         <role-name>Admin</role-name>
         <principal-name>Administrators</principal-name>
         <principal-name>Monitors</principal-name>
     </security-role-assignment>

After this modification you have to restart the WebLogic Server.

That’s it!

, , , ,

Leave a comment

“#PL/SQL: Therefore, whoever binds forever: automate your tests” – I’m a speaker at #DOAG2017

 

Today my presentation is confirmed: “Therefore, whoever binds forever: automate your tests”. Maybe the German title will be the better understood: “PL/SQL: Drum test-automatisiere, wer sich sich ewig bindet!”

I will speak over my experiences with following

  • several PL/SQL testing tools like at least
  • I will give you a demo of this tools and compare their capabilities for
    • architecture
    • branching
    • continuous integration
    • code coverage

If you speak german and you are interested in some of this, you should come and attend DOAG 2017.

, , , , , , , , , , , , , , ,

Leave a comment

#Javaland 2017 wrap up

Yes – I did it again and attend Javaland conference in Phantasialand Brühl.

It was not easy this year to concentrate on the sessions because of the hottest march of the last 100 years. But the quality of the sessions beats the weather. Maybe again my invest in reading the abstracts and filter the sessions before the conference has payed off.

Day 1 Conference

Jens Schauderdocumentation & slides with AsciiDoc, Git, Gradle and Reveal.js

Jens shows in this early morning session how we can version and so diffing and merging our documentation and presentations via text based markup languages. Now I have to puzzle the pieces together for generating my future documents from code. One question remains – what if we have to share the work on the documentation work with non programmers?

Jean-Jacques van Oosten, Chief Digital Officer Rewe Group: The role of the CDO as Chief Product Owner

This presentation shows REWE as on the right way for digitalization. But somehow I did not get rid of the feeling, that this was more a recruiting campaign to get the right people for the job. Have to speak with some of my friends, which are working there in the field. Interesting is the goal to develop all IT itself and do not buy from competitors perhaps. This needs lot of money and resources.

Steffen SchluffSebastian Damm, OIO – Orientation in Objects GmbHMust Feature Branches and Continuous Integration contradict each other?

I know most of the facts already but here these was ordered and coherent derived from literature and practices. Conclusion – with the right tools you can live with the compromise.

Gorge Albrecht, Code Mentor: Readable tests with AssertJ

Unfortunately I come a little bit late, because there is not planned really a big break for lunch. It seems here I have missed the point before: I had hope that the test output is more readable for the tester, not for the programmer. The only disappointing session for me.

Sven Ruppert: Hacking with Mutation Testing

After I have heard Sven’s session for PIT framework last year on Javaland I found not the time till today to work with this. Unfortunately I bring my company laptop which does not allow to download files from maven over the VPN as the company man in the middle webwasher replaces the original certificates. Sven does his best to support me, but I could not solve this in this time. In the evening on my way to home a nameless attendee give me an hint to disable the ssl check in maven locally. So I could test Sven’s solution branch right before midnight successfully!

Carola Lilienthal, WPS – Workplace Solutions GmbH: From Pair Programming to Mob Architecting

A little bit late, I enter this fully packed room. This was announced before via “chili” in the conference planner, so no excuse for me. But as I could not make notes standing on the door in the last row, I think I have here to reread the presentation to give a concrete summary here.

Stefan Schlott, BeOne Stuttgart GmbH: Security in the development team

The presenter gives some hints for small development teams to make it fit for security. Unfortunately most of this are not happened in big companies too. Additional most developers are not interested in this too. In this session first I heard the term required Caretaker on this conference, which should repeated in several sessions.

Mario Mann, Novatec Consulting GmbH: Application Performance Management with Open-Source-Tooling

As session name promises: He show different tools for different purposes of APM including it’s own. Key message: No one tool for all purposes. Interesting some example test of the Javaland homepage.

Martin Thompson, Real Logic: Keynote: Listening to the Design Pressures

The second keynote starts with statement that “non functional requirements” should be changed to “quality attributes”. Further he states that in most cases all new hypes are covers of earlier research. So he recommend to first search for the original ideas. He mapped an example from 1983 and mapped several statements in the content to actual hypes. We have to bring back simplicity and other core techniques, as hardware will not grow anymore as in the past and will not solve our growing requirements. Doing it working and fast together in one step will give other design decisions as doing it separately.

Meet-the-Lib

While most other people try the rides of Phantasialand, I attend this format. Same motto as last year: show me your favorite opensource lib and I show your mine. Not all was interessant for me, but some of this I would never heard, when not here.

  • Lombok: Boilerplate generator, unfortunately not really usable in JDeveloper
  • SPI – Service Provider Interface: interesting, but see no use case at the moment
  • jQAssistant: QA for your code, very interesting open analysis tool, have to read the corresponding missed presentation
  • Form Checker: generic validation and creation of HTML forms for portals, no use case in our framework development
  • Future of Spring: concurrent JEE framework to ADF
  • Jax-RS Hypermedia: maybe later when REST required in our development

After dinner and some drinks I return to cologne.

Day 2 Conference

Elmar Juergens, TU München und CQSE GmbH: How sensible we use our quality analysis?

“From weighing alone the pig will not be fat”. Many analyse it’s code but do little against the results. He shows very entertaining the combination of research and practical steps to wake the interest to change the code. Again you need a Caretaker here. But remember “Management by numbers is not by professionals but by amateurs.

Rustam Mehmandarov, Computas AS: Field Notes of a Command Line Ninja

Lot’s of linux command line tricks, have to attach my notes to every bash call from now. Very entertaining.

Thorsten Maier, Orientation in Objects GmbH: Resilient Software Design Patterns

Good overview, key message: Use it only if you really need it, as it adds new layers to your software stack.

Falk Sippach, Orientation in Objects GmbH: Continous Architecture Documentation

This presentation was a good complement to Jens Schauder from Day 1. Falk shows the “documentation smells” as reasons to use here again markdown based documentation to generate to different target formats and audience. Additional he show some tips and tricks for the used tools.

Stephan Classen, Canoo Engineering AG: Robust, readable Web UI Tests with page object pattern, Geb and Spock

Good presenter, but I’m not really convinced from this wrapper tools (Selenium -> Geb -> Spock) in different technology stacks. Maybe sometime I will try it to be sure?

Frank Pientka, MATERNA GmbH Information & Communications: Load and Performance tests with ‎JMeter or Gatling

The speaker gives a very wide overview of both tools, I have definitely to reread this. Especial interesting is the feature of both tools (and the wrapper Taurus) to reuse selenium tests for load tests. Have to check this out!

Martin Reinhardt, Holisticon AG: Responsive Testing with the Galen Framework 

Presenter shows this interesting framework, but it same a little bit complicated to me. At the moment not usable for me, as it have no IDE Support for JDeveloper.

Thomas Darimont, eurodata AG

I had see some of this in last year “Meet the libs” format. As I had some plans for private application development, this seems the ultimately completest and free AIM solution for me. The speaker had deep understanding of the product and he packed this last presentation full with his knowledge. If you need somebody to set up your system, you should hire him!

Day 3 – Workshops

Gerd AschemannContinuous Delivery of Continuous Delivery

The trainer here has too a deep knowledge of the area and put all of this into this workshop.

We started with the creation of a VirtualBox via Vagrant. The we added script provisioner to install puppet into the box. With another script provisioner we start the installation and configuration of Jenkins via puppet scripts and modules. After that we installed Docker in the VirtualBox via another puppet script. On top of this we automatically create jenkins jobs to start a Docker container containing Maven and building there a simple Java Application. As last step we add an Nexus container to deploy the resulting artefacts.

The trainer provides us with some theory, but most time was practical work. Additional to the described steps we add some configurations for testing the created infrastructure and minimize the creation time of the infrastructure via caching.

The whole time we push our local git repositories to an central instance of the trainer where our environment are build (or not) via a central Jenkins instance.

The extensive workshop scenario, the different operating systems and bugs in the newest version of vagrant make it more difficult to reproduce all steps just in time. But we get all sources and slides to reproduce the scenario later at home.

Despite I have already started an similar scenario at home before, I get a lot of fresh input for my future work here. This workshop was every minute worth.

Conclusion

Same as last year: This was inspiring conference, which gives me a lot of fresh ideas and reminders for some spilled goals for my private and company projects. I think we’ll see us next year!

Improvements for next year

There are some for next year in Phantasialand:

  1. Please deliver stable and fast internet at least for the speakers and the workshops.
  2. Put the labs and commuity workshops out from the exhibition hall to filter the noise from soundchecks and nighthacking.
  3. If you see via the favorites of dukecon, that rooms are full, maybe you can change locations.
  4. For workshops provide big enough tables for comfortable working.

That’s it.

, , , , , , , , , , , , ,

1 Comment

#DEVCAMP17 wrap up

 

Yesterday I attend the annual barcamp DEVCAMP of the DOAG development community. There are mostly developers with SQL, PL/SQL, Forms & Reports, ADF, JET background and some Java, Javascript and APEX developers too. And not to forget: 1 dba and some managers. Here are my summary.

I arrived on evening before just in time for the life cooking event and later at the bar to meet lot of the people I already know and some new too.

The morning begins with a short introduction to the format of a barcamp.

As warm up Jens Schauder starts with an talk about clean code. He begins to compare what the lack of hygiene for Middle Ages mean and the statement that we are in the IT Middle ages. In extreme cases not tested and not understandable software can kills people! So the task for every day is “write an test” and refactor the code to clean code.

Then the barcamp planning starts. In short time three slots are filled up and the really barcamp starts.

img_20170208_105240

Influenced by the starting talk next there was a session related to clean code and how to implement the related processes. You should teach a little bit of the theory behind it. It is essential to decide all rules with the whole team and review the continuous use of it. If the rules are not followed, then first you should ask why this happens and listen for the reasons. Jens Schauder recommends here these books. Maybe you have to adjust the rules to convince all developers. But in extreme cases of deny any of the decided rules you should escalate and delegate this to the responsible manager.

Next session was about how to test heterogeneous applications, especially SQL, PL/SQL and Forms but others too. I mentioned here two possible strategies, which both has it’s tradeoffs:

I noticed here that testing is mainly seen as to check the contract between developer and customer, sometimes useful for regression test but seldom as required for changing or refactoring the code. The participants seems to be overwhelmed by the amount of tools and effort to do automatic testing. Yes – there is a lot to do in advance, which pays off later.

In the next break I get a shortcut for actual requirement for forms change to implement some kind of SSO without additional servers. I get some nice hints and have now to check, if one of this is successful. This are the advantages of networking here.

The we discussed about Alternatives to Oracle Reports. Seems that it depends on your requirements, which tool to choose. Please do an value analysis here. So if you need pixel exact layout or want not pay much money Oracle Business Intelligence Publisher will not your friend. If you search for an active community maybe you should not choose Eclipse BIRT or at least use the commercial version from Actuate. But both of these and Jasper Reports are used from present participants. Further alternatives seem to be SAP Chrystal Reports and Pentaho. And don’t forget to test exotic features like widows and orphans!

After a coffee break we discussed the future of Oracle ADF. The punchline here was: ADF is not dead, but JDeveloper is very buggy. None of the participants does really miss any bit feature, but most struggles with the IDE. It seems, that the problems are bigger, when you work on windows. Maybe some of the product managers should be forced to worked again with windows to see our problems. I have articulated there 2 problems with the ADF community. I we try to raise the acceptance we should not try to extend the focus of the activities to alternative technologies as MAF or JET. Second we should not self call ADF into question on every conference, we should leave this part to the fans of other technologies.

The last session again turns around automatic testing of PL/SQL, some repeating to the described session above.

In closing session most of the participants said, that they want to come back next year. Maybe it is better to put some fixed sessions or workshops, as this is easier to describe to the bosses as “I will go this event, but it is not clear, what will happens there.”

Conclusion

I have learned not so much new things this year, but I attend these time more the known to me technologies. But I get the confirmation, that most of the things we do or plan do to are on the right track.

Unfortunately I observed some people in the sessions, which comes only to search for finding confirmations to reject for itself, what at least the initiator of the topic want to do.

You find more informations in Twitter

That’s it!

, , , , , , , , , ,

Leave a comment

Virtual Development Server: Provide Docker images in docker registry for Oracle via Jenkins

This is a follow up to earlier Virtual Development Server: Provide Docker images in docker registry for Oracle XE Database and others.

We now try to automated build of all needed images as soon as Oracle GitHub Sources changed or we need to build our Vagrant VirtualBox again from scratch.

After we have already installed Jenkins we now install first some needed plugins.

After this we create our Jenkins pipeline via “New Item”.

Here is the script for the moment, later after we have it successfully tested and implemented all builds of images we need, we will put it in it’s own git repository and call it from there. The script is written in Pipeline Model Definition, a config-like syntax for defining Pipelines.

pipeline {
    agent label:'localhost_vagrant'
    stages {
        // Get the actual docker images sourcse from oracle git account
        stage('Get Sources') {
            steps {
                git url: 'https://github.com/oracle/docker-images.git'
            }
        }
        stage('Build Oracle Docker Images') {
            steps {
                parallel (
                    "Database XE 11.2.0.2" : {
                        dir('OracleDatabase/dockerfiles') {
                            // provide the downloaded source (via your oracle account) from the host
                            sh 'if [ ! -f 11.2.0.2/oracle-xe-11.2.0-1.0.x86_64.rpm.zip ]; then cp /software/Oracle/Database/oracle-xe-11.2.0-1.0.x86_64.rpm.zip 11.2.0.2/oracle-xe-11.2.0-1.0.x86_64.rpm.zip; fi'
                            // build the image
                            sh 'sudo ./buildDockerImage.sh  -v 11.2.0.2 -x'
                            // prepare the image for push to the local registry
                            sh 'docker tag oracle/database:11.2.0.2-xe localhost:5000/oracle/database:11.2.0.2-xe'
                            // push to the local registry
                            sh 'docker push localhost:5000/oracle/database:11.2.0.2-xe'
                        }
                    },
                    "WebLogic 12.2.1.2" : {
                        // server-jre is required for Weblogic
                        dir('OracleJava/java-8') {
                            // provide the downloaded source (via your oracle account) from the host
                            sh 'if [ ! -f server-jre-8u111-linux-x64.tar.gz ]; then cp /software/Oracle/Java/server-jre-8u111-linux-x64.tar.gz server-jre-8u111-linux-x64.tar.gz; fi'
                            // build the image
                            sh 'sudo ./build.sh'
                            // prepare the image for push to the local registry
                            sh 'docker tag oracle/serverjre:8 localhost:5000/oracle/serverjre:8'
                            // push to the local registry
                            sh 'docker push localhost:5000/oracle/serverjre:8'
                        }
                        dir('OracleWebLogic/dockerfiles') {
                            // provide the downloaded source (via your oracle account) from the host
                            sh 'if [ ! -f 12.2.1.2/fmw_12.2.1.2.0_wls_Disk1_1of1.zip ]; then cp /software/Oracle/WebLogic/fmw_12.2.1.2.0_wls_Disk1_1of1.zip 12.2.1.2/fmw_12.2.1.2.0_wls_Disk1_1of1.zip; fi'
                            // build the image
                            sh 'sudo ./buildDockerImage.sh -v 12.2.1.2 -g'
                            // prepare the image for push to the local registry
                            sh 'docker tag oracle/weblogic:12.2.1.2-generic localhost:5000/oracle/weblogic:12.2.1.2-generic'
                            // push to the local registry
                            sh 'docker push localhost:5000/oracle/weblogic:12.2.1.2-generic'
                        }
                    }
                )
            }
        }
        stage('Cleanup') {
            steps {
                // clean virtualbox
                sh 'docker rmi --force localhost:5000/oracle/database:11.2.0.2-xe'
                sh 'docker rmi --force oracle/database:11.2.0.2-xe'
                sh 'docker rmi --force localhost:5000/oracle/weblogic:12.2.1.2-generic'
                sh 'docker rmi --force oracle/weblogic:12.2.1.2-generic'
                sh 'docker rmi --force localhost:5000/oracle/serverjre:8'
                sh 'docker rmi --force oracle/serverjre:8'
            }
        }

    }
}

We can wait now the 15 min poll time or start the job manually.

After an “vagrant halt / vagrant up” you can control in your browser, that the created images are persistent in the local docker registry:

docker_registry_persistent

Here you find the source code for this blog.

Here you find more about the topic “Virtual Development Server”.

That’s it.

, , , , , , , , , ,

Leave a comment

Virtual Development Server: Install Jenkins for Continuos Integration / Delivery / Deployment

For automation of all of my processes I need Jenkins in my development server.

First I had installed Jenkins as Docker container via Vagrant Docker provider. But with this architecture it was very complicated to create docker images and run Docker containers on the Docker host, in this case my Vagrant Virtualbox.

So for simplification I have decided to install Jenkins with the provisioning of the Vagrant Virtualbox via a shell provider in my Vagrantfile:

...
  # persistant storage for jenkins
  config.vm.synced_folder "C:\\shared\\virtual_storage\\jenkins_home", "/var/lib/jenkins", type: "nfs", owner: 994, group: 992, create: true
...
  # install jenkins
  config.vm.provision :shell, :path => "add_jenkins.sh"
...

First I map a shared folder of my Windows Host to the Vagrant Virtualbox to persist my Jenkins Configuration.
Then I install Jenkins via the shell script add_jenkins.sh.

sudo yum -y install java
sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo
sudo rpm --import https://jenkins-ci.org/redhat/jenkins-ci.org.key
sudo yum -y install jenkins
sudo sed -i -e 's/JENKINS_JAVA_OPTIONS="-Djava.awt.headless=true"/JENKINS_JAVA_OPTIONS="-Djava.awt.headless=true -Dmail.smtp.starttls.enable=true"/g' /etc/sysconfig/jenkins
sudo service jenkins start

Jenkins requires java, so I install it first. After that I get the latest Long-Term Support (LTS) repository, import it and install Jenkins from there. Then I do some configuration, which I need for my jobs and start Jenkins. An open issue is, that I have to restart Jenkins after Vagrant restart, because the shared folders are unfortunatly mounted after service starts.

After first install you have to enter the credential information from the filesystem as requested, all later installations use the persistent Jenkins configuration, so you can login and start from where you have left Jenkins.

unlock_jenkins

Here you find the source code for this blog.

Here you find more about the topic “Virtual Development Server”.

That’s it.

, , , , , , , ,

1 Comment

Fix “BEA-000394: Deadlock detected” for Weblogic with ADF Applications 12.2.1.2

If you try the newest version 12.2.1.2 (or maybe earlier versions too) you see, that your managed server does not restart, when

  • you have deployed your ADF application(s)
  • you try to restart the managed server with ADF
  • you use intentionally or not shared libraries in your weblogic-application.xml (see example below)
<?xml version = '1.0' encoding = 'UTF-8'?>
<weblogic-application xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                      xsi:schemaLocation="http://www.bea.com/ns/weblogic/weblogic-application http://www.bea.com/ns/weblogic/weblogic-application/1.0/weblogic-application.xsd"
                      xmlns="http://www.bea.com/ns/weblogic/weblogic-application">
  <listener>
    <listener-class>oracle.adf.share.weblogic.listeners.ADFApplicationLifecycleListener</listener-class>
  </listener>
  <listener>
    <listener-class>oracle.mds.lcm.weblogic.WLLifecycleListener</listener-class>
  </listener>
  <library-ref>
    <library-name>adf.oracle.domain</library-name>
  </library-ref>
</weblogic-application>

You run into unpublished Bug 24490566: INTERMITTENT DEADLOCK IN WLS STARTUP DURING ADF-SHARE-DEPLOYED-TESTS.

But no problem, Oracle has created Patch 24490566: INTERMITTENT DEADLOCK IN WLS STARTUP DURING ADF-SHARE-DEPLOYED-TESTS for us for version 12.2.1.2.

Remember to install the patches for all of your developers too!

Remember too, if you want to install more than one patch, there can be conflicts. Please test carefully, if all of your problems are solved for all patches after adding one more patch. Maybe you have to create a merge request in Oracle My Support.

That’s it!

, , , , , ,

5 Comments

#DOAG2016 wrap up

doag_wegweiser

Another DOAG is over and here are my summary.

Day 1

Unfortunately the early train from duesseldorf arrives not before 9:30, so I missed the first morning slot. But on the other hand I this was the slot where I had to decide between 8 presentations, which I’m interested in.

The first soft skill presentation was about creative techniques beyond brain storming. The mentioned 3 techniques was all unknown for me, maybe I give it a try next time.

The next presentation promises a live demo of a forms 12c migration to production. Nothing really new here and remembers me for an “beautify our village” contest. But fairly enough for customers which absolutely want to remain on Oracle Forms. As Translation Hub 12c is not available and Oracle Reports 12c will be the last version, I think 11g will the last version we will use on our side before completing migration to ADF.

I attend two presentations, which shows Auraplayer on top of Oracle Forms to provide mobile applications via Oracle Mobile Cloud Service (MCS). I understand the MCS thing, although I find it very complicated to use. But put another layer on complicated and most slowly Oracle Forms to create web services for mobile applications seems no use case for me. I doubt that the recording of web services works really well for complicated forms which switch the displayed items and canvases depending on the selected data. All samples was very simple forms, but the response time of the resulting mobile app was not acceptable for me.

The presentation about continuous integration in APEX give me some new ideas for maybe replacing our actual deployment processes for PL/SQL in the future. Will have a look at the mentioned sql-maven-plugin, when we go for Maven for ADF too. Will check Rspec as BDD framework to run PL/SQL unit tests. Promising is the feature to tie together Selenium tests and data checks in the same Rspec test definition. One drop of bitterness remains, as the presenter had no idea how to solve a merge conflict. Seems that this was only a proof of concept and not really used daily there.

Best practices for PL/SQL performance was a good reminder, what I all know about this topic but use to seldom. But will use some of the configuration things like native compilation of the system packages or PL/SQL optimization level in my docker instances for automatic testing to optimize test times.

SQL tips and tricks was a sovereign presentation, but shows me again that for several Oracle database releases only little improvements for the developer comes with.

Day 2

In session “Standing at a Crossroads, ADF and JET” Duncan Mills clearly show the differences between Oracle ADF and Oracle JET. Running gag: The “Schnecke Chart” for quick decision between both. Will look, if I can reuse this for other decisions.

Next I tried an meeting at the Unconference about continuous integration for PL/SQL. But the most discussions really goes about the automation part. Most participants missed the most importantly definition of continuous integration: ” … continuous integration (CI) is the practice of merging all developer working copies to a shared mainline several times a day.”. Unfortunately I realize now that in the German Wikipedia translation this basis is completely softened: “developer should integrate as often as possible”. Not really new for me here, hope I have told something useful to the others.

“Extending Oracle ADF BC Models to REST, Cloud and Mobile” was a good time to remember what’s now possible as it really available and we now try another migration away from ADF 11g to the hottest version. I have seen this in the last conferences already.

Have read “Deep Dive Into Oracle ADF Transactions: Advanced Techniques” in the web already, but the information was a little bit clearer after seeing Eugene Fedorenko live.

Something completely different: “Free Load Testing Tools for Oracle Database”. I had known Swingbench before. SLOB is completely new for me and did not seen Apache JMeter JDBC test in this context before. Will test the last maybe in the future.

“Top 10 stumble stones for ADF 12c” was very technical but cool, good to have Markus Klenke at our site while our coming migration next weeks.

“Faster, Better, Cheaper: Oracle ADF Development in the Cloud” – with lot of problems in the demo part. Maybe really because of the DOAG WiFi. Quick Tip for all presenters: as I know at DOAG conferences there is ever an laptop available with LAN, try to put it in your laptop! For the content: DCS seems to restricting to much. I prefer Jenkins over Hudson. No push option directly with commit i JDeveloper etc.

Masking test data is very relevant in our banking busines, so I attend “Oracle Data Redaction live in Telko”. Seems that there are many fall knits here, as this doesn’t seem to solve all requirements out of the box.

Before the evening keynote Günter Stürner and Jürgen Menge was awarded, congratulations to both.

Following keynote “Hacking for managers” was very entertaining, if you are German then you should look on youtube for the presenter.

Now we are not ready for the evening party, we start the yearly German ADF Community meeting. Interesting part was a presentation of the Tools Statement of Direction, which again clearly differentiate between ADF, MAF, JET, MAX and ABCS. We get the clear statement ADF is standard for SAAS and JET for PAAS development at Oracle. In my opinion Oracle miss to communicate this in other presentations and miss to further present ADF in the public. A lot of questions of the present Community members was answered here too. Later we eat together with some of the product managers and community members and discussed topics for future presentations.

Later I go back to my hotel to remotely patch our fresh ADF 12.2.1.2 Server with patch for critical escalated bug to start our planned migration for ADF 11.1.1.5 next week. More later in a separate post.

Day 3

In “Indiana Nimphius und der JavaScript Tempel des Todes” Frank shows us his view on the JavaScript Universum, maybe we can really not avoid this in the future on some business cases. But for the next time we will stay on strongly supported ADF and will reduce JavaScript usage there to the minimum.

Thank god the roll-out of SQL Developer 4.1.5 is not started already in our company. Will look how fast I can add the free PLSQL/COP SQL Developer Plugin of Philip Salvisberg from Trivadis to the silent install process. He showed in “Fighting Bad PL/SQL” the theory and the practice how his static code analysis tool for PL/SQL does work. Depending on the outcome and the price maybe we will later implement in continuous integration.

The keynote “Big Picture of digital revolution” wakes the expectation to us to take responsibility for the effects of digital revolution to the society and makes me very thoughtful.

After our hopefully successful migration to ADF 12.2.1.2 I want to change our deployment from ANT to Maven, so I attend the presentation “Migration Maven Possibilities”. This gives a good reminder what to do at that point.

Very interesting presentation, which really does not deserves so few visitors, was “Home Lab Setup and Familiarization of E-Business R12.2”. My time with E-Business Suite is over 15 years away, so I was surprised that I recognized much of the components. I think I’m sure not to install E-Business Suite for the moment, but lot of the instructions for the Home Lab Setup could I reuse for other technologies. Some concepts for the E-Business Suite, eg. double filesystem for reducing offline time for patching, are very interesting.

With last 2 presentations I enter again the world of ThickDatabase paradigm. We ever know that executing business in the database seems faster as in the middle tier, but we had no proofs. Now Toon Koppelaars from Oracle itself give us this with a deep analysis. The question remains – will the other Oracle departments will learn from this?

day 4

Tomorrow I will be trained in Continuous Delivery with Oracle Fusion Middleware and Oracle Cloud. I will release another blogs, when implementing some findings from this training.

conclusion

As you see I switch often between technologies. This happens not ever voluntary. Normally I interested more in the development like ADF, MAF and JET. This year these presentation are unfortunately distributed over the conference. So I have discussed this today with the DOAG executives, they will improve this next year.

Nevertheless I get a lot of ideas from presentations in technologies I do not or not regularly use, so sometime I intentionally choose from such interesting presentations.

A recommendation for all conference visitors. Read the abstracts before you go to a conference and maybe before you book! You avoid disappointments because of wrong expectations. Often the title has the only purpose to be accept the presentation and has little to do with the real content. Abstracts give you often a better idea about the content.

That’s it!

, , , , , , , , , , , , , , , ,

1 Comment

Virtual Development Server: Provide Docker images in docker registry for Oracle XE Database and others

For later creation of containers as needed in the deployment process we have first to build docker images whenever we do not find suitable one in docker hub.

For most of my development I need a oracle database. On the internet you can find a lot of Dockerfile’s for this, but no ready image. This is because for running a database you have to accept a license and this happens when you download the installation software. For private testing this license often is free.

The basic cycle for all of this images is the same on my side:

  1. Find a Docker image
  2. If not found download required Software and build a Docker image
  3. Build the Docker image
  4. Push the images to private docker registry or the public registry DockerHub

Fortunately Oracle provides recently on GitHub a lot of working Dockerfile’s for building different product images. The creator Gerald Venzl desribes in this blog how to create an Oracle EE image.

Following you find an example for building Oracle XE 11.2.0.2 Image.

First you have to clone the git repository to a directory of you choice, I choose C:\shared\scmlocal\docker-images.
Put the downloaded installation source oracle-xe-11.2.0-1.0.x86_64.rpm.zip in the subdirectory C:\shared\scmlocal\docker-images\OracleDatabase\dockerfiles\11.2.0.2 as described in GitHub.
Next we have to map this directory to our virtualbox via following vagrant instructions. Additional we prepare port forwarding for accessing later the database from our host system.

...
  # Port Forwardings for:
  # - Oracle database port
  config.vm.network "forwarded_port", guest: 1521, host: 1521
  # Oracle Application Express (APEX)
  config.vm.network "forwarded_port", guest: 8080, host: 8080
...
  # map Oracle Docker Images installation path
  config.vm.synced_folder "C:\\shared\\scmlocal\\docker-images", "/docker-images", :mount_options => ["dmode=777","fmode=777"]
...

After that restart the virtualbox and connect to it via

vagrant halt
vagrant up
vagrant ssh

In the output you can see this mapping.

...
    default: 1521 (guest) => 1521 (host) (adapter 1)
    default: 5000 (guest) => 5000 (host) (adapter 1)
    default: 8080 (guest) => 8080 (host) (adapter 1)
...
    default: /docker-images => C:/shared/scmlocal/docker-images
...

Now you create build the image with the Oracle XE (-x) database version (-v) 11.2.0.2 via the provided scripts from oracle. -i skips the MD5 checksum check, as the checksums are not checked in as UNIX text format and so does not work in our vagrant on windows / oel virtualbox combination.

cd /docker-images/OracleDatabase/dockerfiles
./buildDockerImage.sh -v 11.2.0.2 -x -i

Now you have to wait some time. When the process is ready, you get at the time of this writing an image with the binaries installed.

docker images
REPOSITORY          TAG                 IMAGE ID            CREATED              SIZE
oracle/database     11.2.0.2-xe         624d78c89ccb        About a minute ago   1.203 GB
registry            latest              c6c14b3960bd        5 weeks ago          33.28 MB
oraclelinux         latest              1988eb5b3fc6        6 weeks ago          278.2 MB

Next we tag this image for pushing to the local docker repository, which we have created in the first blog of this series and is up with every vagrant up of our virtualbox. After that we push it to the registry.

docker tag oracle/database:11.2.0.2-xe localhost:5000/oracle/database:11.2.0.2-xe
docker push localhost:5000/oracle/database:11.2.0.2-xe

In the output we see the succesful push.

The push refers to a repository [localhost:5000/oracle/database]
345dfbce8859: Pushed
dd4b46d40894: Pushed
11327e04fac0: Pushed
11.2.0.2-xe: digest: sha256:82058ca2c5c201ff46471a8db4d9569da9869d420727595884050f94dad0ec85 size: 955

Now we will check, if we can create a Docker container from our image in the registry and the database is working in it. Therefore at first we delete the oracle/database and oraclelinux images from our virtualbox. Then we create the container.

docker rmi 624d78c89ccb -f
docker rmi 1988eb5b3fc6
sudo docker run --shm-size=1g -p 1521:1521 -p 8080:8080 --name="db11.2.0.2-xe" --restart unless-stopped localhost:5000/oracle/database:11.2.0.2-xe

In line 3 we pull the image from the local registry, start a container, open the ports 1521 and 8080 to the virtualbox and give the container a name.

Unable to find image 'localhost:5000/oracle/database:11.2.0.2-xe' locally
11.2.0.2-xe: Pulling from oracle/database

10ec637c060c: Pull complete
1902f66bc90b: Pull complete
dd8942200fe1: Pull complete
Digest: sha256:41ce1015c8f68ce063e855e7afa96000ededeef1e197e2181bc04c9577d73a1b
Status: Downloaded newer image for localhost:5000/oracle/database:11.2.0.2-xe
ORACLE AUTO GENERATED PASSWORD FOR SYS AND SYSTEM: 9f4e9a733e7983bc

Oracle Database 11g Express Edition Configuration
-------------------------------------------------
This will configure on-boot properties of Oracle Database 11g Express
Edition.  The following questions will determine whether the database should
be starting upon system boot, the ports it will use, and the passwords that
will be used for database accounts.  Press <Enter> to accept the defaults.
Ctrl-C will abort.

Specify the HTTP port that will be used for Oracle Application Express [8080]:
Specify a port that will be used for the database listener [1521]:
Specify a password to be used for database accounts.  Note that the same
password will be used for SYS and SYSTEM.  Oracle recommends the use of
different passwords for each database account.  This can be done after
initial configuration:
Confirm the password:

Do you want Oracle Database 11g Express Edition to be started on boot (y/n) [y]:
Starting Oracle Net Listener...Done
Configuring database...Done
Starting Oracle Database 11g Express Edition instance...Done
Installation completed successfully.

SQL*Plus: Release 11.2.0.2.0 Production on Tue Sep 6 09:41:22 2016

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production

SQL>
PL/SQL procedure successfully completed.

SQL> Disconnected from Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production
#########################
DATABASE IS READY TO USE!
#########################
db_recovery_file_dest_size of 10240 MB is 0.98% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
Starting background process CJQ0
Tue Sep 06 09:41:19 2016
CJQ0 started with pid=26, OS id=759
Tue Sep 06 09:41:24 2016
XDB installed.
XDB initialized.

In line 9 you can see the autogenerated passwords.
When line 46 appears, you can connect to the database.
With following command you can change the passwords of SYS and SYSTEM:

sudo docker exec db11.2.0.2-xe /u01/app/oracle/setPassword.sh LetsDocker

You see following output:

SQL*Plus: Release 11.2.0.2.0 Production on Tue Sep 6 15:49:34 2016
Copyright (c) 1982, 2011, Oracle.  All rights reserved.
Connected to:
Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production
SQL>
User altered.
SQL>
User altered.

As long as you not remove or manually stop the container, it restarts with the virtualbox now via:

vagrant halt
vagrant up

At the moment you cannot make your database instance data persistent for this container, I have opened an issue for this, but Gerald Venzl has promised to to implement this later.

Here you find the source code for this blog.

Here you find more about the topic “Virtual Development Server”.

That’s it.

References:

, , , , , , , , , , , ,

1 Comment

%d bloggers like this: