The project is really aimed at documentation more than temporal XML.  Besides, when I talk to people about the “temporal XML” project I’m working on I get nice blank 1,000-mile stares and they promptly change the subject.  I think people would understand what a specialized “document processor” is.  The project is really about documents that are made mostly out of subdocuments with special tagging so that the subdocuments can be pulled into specialized reports.  The subdocuments can have database-like fields associated with them.  Everything is fully versioned.  The subdocuments and their associated fields are stored in a database so they can be pulled together into documents as alternate views using queries for special reporting needs.  This is useful for things like project management and requirements management.  The documents and alternate view documents can also be extracted from the database and put into formats like .PDF and xhtml so the data can be viewed easily over the Internet.

So now I have to change the name over in Assemebla where the source is stored, and I need to change the names and internal documentation from “maria” to “docsmith” in the source code.  Oh well. I needed to go through and clean up the code anyway.  Lots of documentation is needed, as well as more complete test cases.

My previous experience with Oracle was as a QA tester for a software product that used Oracle 9i and 10g for data storage.  I never used Oracle SQL. I just installed and set up Oracle on a system, installed the software product, and did blackbox testing on the software product.  Taking the exam helped fill a bunch of gaps in my knowledge of Oracle.

For my exam prep I used two exam preparation books.  One was Steve O’Hern’s SQL Certified Expert Exam Guide.  This book covered most, but not all, of the topics that were on the exam.  To study with this book I went through it once and took all the quizzes at the end of the chapters, writing down my answers on paper and then checking and saving the results.  After finishing the book I took all of the quizzes a second time, again saving the results.  Then I went over all of the answers I got wrong and made sure I understood the concepts.  Then I took the example test on the disk that came with the book.  The example test was a close approximation tothe actual Oracle exam.  The example test gave feedback on which questions I got wrong.   I reviewed the questions I missed and took the same example test a second time.

I also went through another exam prep book for the 1z0-051 exam, and looked for blogs and other Internet articles on the 1z0-047 exam.

Like I said, I passed with flying colors.

The one of the things missing from the exam prep books was the “q()” operator that is sometimes used for strings that contain single quotes.  Fortunately, one of the blog articles I found had covered the topic, so I got a question correct on the test because of my web research.

Some of the questions on the exam were almost impossible to get right because the questions were poorly worded.  When reading the questions I could see that one answer would be correct, but the test really wanted a different answer.  Fortunately, O’Hearn recognized this and gave sample questions that were of the same character, so when I took the real test I was able to give the “correct” answer, even though I knew that the real answer was a little different.  Also, the passing level of 66% correct seems a little generous at first, but in reality it helps compensate for the number of ambiguous questions.

The test isn’t perfect, but in the long haul it does what it sets out to do, which is to give us software guys a vehicle for upping our Oracle SQL skills and it gives employers and potential employers some objective measure of our SQL skills.  In my case I have used SQL to a small degree while working on mainframes, building DB driven web sites, and testing commercial software.  I would say that my knowledge of the details of SQL, especially with respect to Oracle has increased dramatically.

Was it worth the $195, plus two certification books?  I can resell the books and get most of my money back, so the total cost will be about $230.  I think the price is worth it.  I can now say that I am an Oracle certified SQL expert.

I just put everything into Assembla if you are looking for code examples.

Look at <https://www.assembla.com/code/mariaxmldb/subversion/nodes>.  

Open “Show revision log”.  

Open Revision 17 (with timestamp “2010/04/19 20:31″).  

Get the components. 

Everything works for me as-is.  I put the config files and modules in my user root on my Mac OS X system.  I run Sedna on Windows 2000 running on Virtual Box.  I run the components out of Eclipse. 

Review my notes in my earlier posts.

Good luck!

I tried to write an Xquery function to execute updates, and I put the function into a module.  It wouldn’t load. 

Updates don’t seem to run from functions in Sedna.  It goes against the Xquery syntax.   Apparently, updates are an extension to Xquery, not an informal part of the syntax, so they don’t load.  

Updates have to be run like DDL statements; outside the Xquery mechanisms.

This presents a security problem. 

With SQL you can update the database using functions and passing parameters into the function.  That clears up some potential security issues, especially since SQL languages have built-in commands that prevent function values from being executed as SQL. 

With Sedna’s Update extensions there is no such built-in commands.  Any application using Sedna has to edit user data for security issues.

The single box examples all use localhost as the Sedna server.  They also bring in a module from a file on the file system.  When I tried this on my systems I found an issue with this.  It isn’t clear from the Sedna documentation which box should have the file stored on it.  

I tried to load a module from the Sedna command prompt and had no problem.  Everything went well.  I copied the commands from the Sedna command prompt to a Java class to load the module from the API.  This failed.  A little trial-and-error showed me how to do it.  The file should be stored on the same box that is running the Java app, or at least on a system that the Java app box can access. 

Here is the contents of the module file that I named ‘maria_functions.xquery’. I stored this on my Java app system in ‘/Users/smoo/maria_functions.xquery’.

module namespace math = “http://mariatxmldb.org/math”;  

declare variable $math:pi as xs:decimal := 3.1415926;  

declare function math:increment($num as xs:decimal) as xs:decimal {  

    $num + 1  

};  

 declare function math:square($num as xs:decimal) as xs:decimal {  

    $num * $num  

};

Here is the main part of my test code that loads, runs and dumps it.  Sorry it isn’t clean, but I just worked this out and haven’t cleaned it up yet.

public int setUpUserDB () throws MariaException  {

if (log.isDebugEnabled()) {

  log.debug(“”);

}

try {

    SednaStatement st1 = sednaConn.createStatement();

    sednaConn.begin(); 

    try {

        st1.execute(“DROP MODULE ‘http://mariatxmldb.org/math’”);

        sednaConn.commit();

    } catch (DriverException e) { }

    sednaConn.begin(); 

    try {

        st1.execute(“LOAD MODULE ‘/Users/smoo/maria_functions.xquery’”);

    } catch (DriverException e) {

        e.printStackTrace();

        throw new MariaException(e, -1, “”);

    }

    sednaConn.commit();

    DBOp dbOp = new DBOp(dbConn);

    try {

        dbOp.verifyDocumentCreationInCollection(conf.getLogger(), dbConn, 

                ”http://maria_functions.org/math”, “$modules”);

    } catch (Exception e) {

        e.printStackTrace();

    }

    sednaConn.begin(); 

    boolean gotAResult = 

    st1.execute(“import module namespace math =

        ’http://mariatxmldb.org/math’;math:square($math:pi)”);

    if (gotAResult) {

        SednaSerializedResult pr1 = st1.getSerializedResult();

        String item;

        item = pr1.next();

        int count = 1;

        while(item!=null) {

             System.out.println(count+” item: “+item);

             item = pr1.next();

             System.out.println(“===================”);

            count++;

       }

   }

  sednaConn.commit();

  sednaConn.begin(); 

  st1.execute(“DROP MODULE ‘http://mariatxmldb.org/math’”); // Clean up. 

  sednaConn.commit();

} catch (DriverException e) {

    e.printStackTrace();

  } finally {

      try {

          if (sednaConn != null) sednaConn.close();

      } catch (DriverException e) {

          e.printStackTrace();

      }

} 

return 1;  // stub

}

The module was stored on my Java app box, which is Mac Book.  The Sedna box is virtualized in Virtualbox and it is running in Windows 2000. 

When I run the app in Eclipse, I get a return value of the math operation of “9.86960406437476″.

Hope this helps.

I find it interesting how much code is not directly related to the end-product.  Easily, the majority of the product code in Java is code for exception handling.  The old batch mainframe programs I used to maintain were similar.  Most old COBOL code is devoted to validating data and validating and verifying data operations.   Java is no different.  Only the syntax has changed.

Most of the overall Java code is for unit testing.  As I write a new Java class I also have to write the JUnit tests.  The JUnit test have more lines of code than the class being tested.   So less than 1/4th of the code that is written is actual product code.  The rest is error handling and V&V.

Sedna is painful to pick up, at least at the start.  Some small, but critical, bits of information aren’t well documented.  Sedna uses a non-standard construct for referencing documents in collections.  It would be nice if they documented this better.  Standard document references look like this: 

doc(‘some-document-name’)

To reference a document in a collection, Sedna added a second parameter for the collection, like this:

doc(‘some-document-name’, ‘some-collection-name’)

I spent about half a day figuring this out and figuring out how to work with it. 

Some other useful things I picked up that also weren’t documented, or at least, weren’t documented clearly.

To get all of the documents and collections use ‘$document’ in the doc() function.  ’$document’ is a  special variable for listing all of the documents and collections.  For some reason this function returns all of the collections as well as the documents: 

doc(‘$documents’)

To get just the collections use ‘$collections’ in the doc() function:

doc(‘$collections’)

To get just documents that are not in a collection is pretty easy:

doc(‘$documents’)/documents/document

To get a list of collections use:

doc(‘$collections’)/collections/collection

One notable problem I ran into was figuring out how to get a list of documents in a given collection.  The easiest way I found so far is:

for $x in doc (‘$documents’)/documents/collection where $x/@name=’the-collection-name’  return $x

[UPDATE: April 27, 2010:  A simpler query to get a list of all documents in a collection:

doc('$documents')/documents/collection[@name='the-collection-name']/*/@name/string()

]

Determine whether a specific document is in a given collection is a little more complex:

count(for $x in doc (‘$documents’)/documents/collection where ($x/@name=’the-collection-name’ and $x/document/@name=’the-document-name’) return ‘something-to-count’)

If the return value is ’0′ the document is not in the collection.  If the return value is ’1′ the document is in the collection.  Let me know if you’ve found an easier way to do this.

Some of the examples and documentation I found on the Internet for literal strings used single-quotes, and others, such as the XML:DB documentation, used escaped double-quotes.  It took a little digging in the W3 XQuery specs to figure out that both single- and double-quotes work.  In my Java code I don’t need to surround literals with escaped double-quotes. 

The error handling took a little researching.  I looked in the Sedna source code.  Sedna has almost 600 unique error codes.  My problem in designing the Maria Temporal XML DB system was figuring out how to handle these.  I finally figured out that I need to make the Maria the application  layer isolate the Sedna errors and translate them into a set of unique Maria exceptions.  If I move to a different database, such as DB2, I can do so more easily because the DB error codes are isolated from the application.  This approach also keeps the Sedna error codes away from the end users as much as possible, and allows me to write error messages that are more meaningful to my end users. 

Everything I’ve done is in <https://www.assembla.com/code/mariaxmldb/subversion/nodes/trunk?rev=12> if you want to look a little closer. 

The main things I have accomplished to-date are:

1) Getting a better handle how to use log4j.  I think I’ve set logging up fairly well in the Maria system.  I see some room for expansion in the near future, especially as I start designing for an online system with multiple users. 

2) Getting a better handle on how to use JUnit.  There are still some mysteries to it that have to do with the magic of Java more than JUnit itself.

3) Figuring out how to do basic things in Sedna’s version of XQuery.  Like, how to access documents in collections.

4) Getting a better handle on Java in general.  I spent some time digging into log4j and how to use it.  Log4j gave me a better understanding of Java.

5) Figuring out how to use Eclipse with a remote Subversion repository. 

In general I spent the last week and a half just coming up to speed on the basic technology.  I’m gaining some traction and now I can spend more time actually developing the meat of the Maria Temporal XML Database.

I set up a basic framework in Java last week using Eclipse for creating a Java connection to a Sedna installation running on a Windows server running in VirtualBox.  I set up the start of a jUnit testing environment.  The jUnit tests were able to connect to Sedna and run Xqueries to update and query the database.  With that much functionality complete I can move on to actually building the tXML database.  Most of the design for the back end is pretty much complete, so I don’t need to spend a lot more time designing.

I’m using a free Internet service to handle the source code.  You can view the SVN repository here on the minute chance that you are interested:

http://svn2.assembla.com/svn/mariaxmldb

I’ve sort of got Eclipse figured out.  I can do basic stuff with it like building packages and running unit tests.  I added an SVN component called Subclipse.  Installing Subclipse taught me something.  There are two versions of Subclipse, 1.4 and 1.6.  I needed the version that match the version of Subversion running on the free Subversion hosting site, Assembla.  There was no documentation on Assembla about which version they use, so I had to guess.  I made the mistake of starting with version 1.6 of Subclipse.  1.6 was the wrong version of Subclipse, so I had to change to 1.4.  That was a minor headache because I had to manually find and uninstall all of the 1.6 components before I could install 1.4.  Next time when I have to guess like that I’ll try the earlier version first.  If it doesn’t work I can just upgrade to the newer version and save the unistallation headache.

Getting Subversion figured out was a little difficult.  I never used it before.  I spent a couple of hours trying to figure out how to get data to and from the web server.  I went through a about six tutorials, including the semiofficial one from red-bean, Version Control with Subversion.  None of them really helped.  They tried to teach the reader how to use Subversion by working with a preinstalled data set on a local server.  I was trying to create a new data set on a remote server.  I finally found a decent tutorial article written by Jared Richardson that helped a bit.  Then I found a good cheatsheet of commands at yolinux.com that explained a little better what the commands were doing.  I’m like a lot of techies.  I don’t want the user guide.  I want the well written, well documenting, raw technical manual.  ”Here are the commands, here are the options, and here is what they do.”  If a reasonably competent programmer can’t figure it out using primarily the technical manual, the technical manual should be revised.  If I have time I’ll do an improved version of the yolinux cheatsheet.  That is stuff I need to know for the Maria project anyway.

I eventually figured out that to get started with Subclipse I had to put the source documents onto the Assembla repository before I could use them in Eclipse.  After the documents were on Assembla I could use Subclipse to bring them into Eclipse.

One of the tricky things I found was getting an Eclipse Java project to work with a Subclipse repository.  It took a bit of doing. In Eclipse I first created a generic project from my remote SVN repository.  Then I created a second project, this one a Java project.  While I was creating the Java project I added source links from the SVN folders in the O/S into the new Java project.  When everything was set up correctly I had a working Java project in Eclipse with an SVN repository in Eclipse.  Now I can do all of my edits, checkins, etc. from within the Eclipse environment.

What a day.

I decided to use VirtualBox for my virtual machine.  VB is a stable and robust VM. 

Oracle runs on Ubuntu server, so I went with a Ubuntu server image. 

Fortunately for me (and for you too, I assume), Eoin Woods has already written up an excellent How-To on his blog Viewpointsperspectives and the rest for installing Oracle 10g Express on Ubuntu server on VirtualBox on OS X.  I recommend that you read through my notes here and then follow Eoin’s article to do the install.  My documentation here contains caveats, updates and variations of what Eoin covered.  

One of the problems I ran into when walking through Eoin’s How-To had to do with Ubuntu versioning.  Eoin used the 8.1 Ubuntu server.  I am using the current version which is 9.10.  However, there are two builds, one for 64 bit systems and one for 32 bit systems.  I made the mistake of getting the 64 bit system.  I assumed that because my Mac has a 64 bit architecture I would be able to use the 64 bit Ubuntu server.   Oracle 10g Express is set up for 32 bit architecture and complained about the 64 bit architecture when I tried to install it.  There may be a way to make it work, but the way wasn’t obvious to me, so I deleted the image, downloaded the 32 bit server, created a new 32 bit Ubuntu VM for it, and went from there.

I created my Ubuntu server image with Bridged networking instead of NAT which Eoin did.  Eoin used a NAT network adapter in his network configuration as a way to keep connection issues to a minimum.  I want to be able to access the server directly from a real Windows computer in my home network.  Some software tools for Oracle are Windows based and I may want to use those.   So, anyway, I went with Bridged.  

With Bridged networking, the server is a part of the home network just like a real machine.  With NAT networking, only the computer that the VM is running on has direct access to the VM.  The main down side of Bridged VMs is that they can have problems keeping a consistent IP address after the VM is shut down and restarted.  The home router can assign it a different IP address arbitrarily, which is something you don’t want when you are connecting to a server by its by IP address.  I don’t believe NAT has the same problem with IP addresses.  So, I went with Bridged networking.  

Eoin installed SSH as one of the last things he did.  I installed SSH when I installed Ubuntu.  There is a checkbox in one of the Ubuntu 9.10 installer options that will take care of that for you at install time. 

The default swap space for Ubuntu is not large enough for Oracle, and it needs to be enlarged.  There are at least two ways of doing this.  Where Eoin created a larger swap space using a few commands, I created a larger swap space at install time.  That wasn’t completely intuitive and it took me a couple of hours to figure out how, but it only takes a minute now that I know how.  I did the following steps to set up a swap space on an 8Gb hard disk:

1) When given the Disk Partition options appears during installation, choose the Manual option.

2) Select the Guided Partitioning option.

3) Select the SCSI1 … option.

4) Yes, create the new empty partition table.

5) Select the pri/log option.  It will all be FREE SPACE.

6) Create a new partition. 

7) Make the new partition all of the disk space except for 1.1Gb. 

8) Make the new partition Primary.

9) Make the new partition start at the Beginning.

10) Keep the defaults for the Partition Settings, and select Done setting up the partition.

11) Back on the main partitioning page, select remaining 1.1Gb of pri/log FREE SPACE.

12) Create a new parition.

13) Use the remaining 1.1Gb of free space.

14) Make the partition Primary. (I don’t think it really matters if it’s Primary or Logical.)

15) Select the Use as: option.

16) Select swap area.

17) Select Done setting up the partition.

… and Finish Partitioning … , Write the changes to disk, and continue the install from there.

Setting this up at Ubuntu’s install-time works, but Eoin’s four step, post-install approach seems simpler than the installer-based 17 steps.  I’m not an expert on Linux.  I wonder, what happened to the original swap space in Eoin’s system after he set up the new swap space?  Is that just unused disk space now?  I don’t know.  Anyway, both approaches work fine for our purposes.

One thing I did different was to update Ubuntu after installing it. Right after the installer finished I did:

$ sudo apt-get update

That brought the software in the system up-to-date.  

After Ubuntu was installed and set up I took a snapshot, so I have a clean system to revert back to if something went wrong when installing Oracle.  After Oracle was installed I took another snapshot so I have a clean system to revert back to if I totally hosed the system.

A few other points for us newbies. 

I had problems using vi in this setup.  I couldn’t get the “: ” command prompt to come up by pressing escape.  I found that pressing escape and then ‘Q’ brought up the command prompt.  I don’t know what the story is here.  Maybe its just a lack of knowledge on my part, but I thought that pressing the escape key would bring up the prompt. 

Getting an ssh connection from my Mac took a little bit of figuring out, because I haven’t used ssh before.  It’s easy.  ssh allows you to connect to another machine as a user of that machine.   On my Mac I am user scott.  On my Ubuntu server I’m user fred.  My Ubuntu server has IP address 192.168.2.105.  To get an ssh connection from my scott user account on my Mac, I just opened the command prompt on my Mac and entered the command:

$ ssh fred@192.168.2.105

… not at all difficult once you know how.

Viewing more than a page of output in the terminal on the Ubuntu server took a little time to figure out.  To see previous lines, just use  Shift + fn + (up arrow or down arrow) to scroll up or down.

Good luck!

… in theory …

In reality, there are some problems with CSS.  

Not all browsers support CSS the same way.  We still have to write special codings for different browsers.  To create the special codings we have to take advantage of certain nuances of browser implementations. 

CSS doesn’t always work as advertised.  I don’t create a lot of web sites from scratch, but I have yet to implement a style sheet that worked correctly.  The problem I keep running into is that I expect CSS to work as documented. Silly me.  There always seems to be something I do that the browser I’m using doesn’t support, or for some mysterious reason the CSS styling simply doesn’t work. 

Add to those issues the problem with caching and CSS is a pain in the butt to work with.  

When I supported COBOL mainframe systems I always knew that my COBOL codings would work as documented.  That’s one of the reasons why COBOL is great as a programming language.  It may not be the most powerful in a technical sense, but mainframe COBOL is extremely powerful from a business sense.  You don’t spend a lot of time figuring out that the feature you are implementing doesn’t work on your mainframe.  The only exception I ever saw to this was when a programmer tested a program in a third party debugger and implemented it without the debugger.  It turned out the debugger did sorts slightly different than the regular compiler.  (The debugger used a different library than the regular system library.)  I believe that problem was fixed in more recent versions of IBM mainframe COBOL. 

But I’ve come to hate CSS.  I like languages that work as advertised and CSS ain’t that.