Blog
what did i learn today
I4i battling Microsoft?

...or software patent troubles again

The weirdest news i read yesterday: I4i is not out to destroy Microsoft? In the article is described that some small company, i4i, has apparently won a case against Microsoft, claiming the Microsoft Word violates a patent they hold. More specifically: patent5787449. The Texas judge ruled that Microsoft has to pay a fee/fine of 200 million dollar, AND that Word versions 2003, 2007 and future versions can no longer be sold. And the problem only arises when clients or customers use a feature called "custom XML". What? Let's rewind here for a bit. What does that patent actually say? Let me summarize it for you. It describes two things: 1) it describes a system where the content is seperated from the actual representation, and 2) using a dialect of SGML. Now note: this patent was filed in 1994, before XML even existed (only since 1996), and the patent was finally issued in 1998. I tried to read the patent-document as some kind of specification, but i am not even sure what it is actually protecting. An algorithm? The idea? Because the idea of seperating content and representation is a common idiom. We all do it. Websites do it. We use css all the time. We use templates in Word ... The idea of using metacodes (like Title, Chapter, ...) and then looking up their definition the replace in the actual document (this is the example from the patent). So it must have something to do with the specific implementation: they use an XML file. Now comes the really scary part. XML is really powerful, and Word is using a set of XML files, packaged in one zipped archive as their file format since Word 2003. This is really awesome. This is more open than any binary format. Now anyone who can read can actually create or process Word-files. Microsoft calls it Open Office XML or OOXML, but that doesn't really mean the standard is really open. Only Microsoft can amend the standard, and the other big standard, OpenDocument (ODF), is the fileformat originally used in OpenOffice.org (you see the deliberate confusion in the naming?), is totally different, not supported. A missed opportunity. But that is another discussion all together. Now, i4i has no problem that Microsoft uses an XML file, just as long as the format is fixed. Meaning: you specify which tags you can read, are defined (for instance using DTD), and those tags are fixed. But Word allows you to add Custom XML, you can add your own fields into any Office document. And i4i claims that is the problem. So now i am lost. This has nothing to do with seperating content and presentation (that was what the patent was about, right?). It could have something to do with how the extra tags are added, the custom XML. If they would use a lookup-table, to identify the custom tags, then they use a technique that is described in that patent. But hey, wait, does that mean you can not use custom xml tags anymore? I have used tons and tons of xml, with no predefined DTD, so that the file format is extensible, and users can add their own extensions, and keeping the definition of the tags seperate. In my personal opinion, and i think i am not alone, Microsoft is falsely convicted. There is no technological ground this patent can hold. This all comes back to the old software patent discussion. What is patentable and what is not. It seems so unnatural to patent software, algorithms, they are the building blocks upon which we learn and build the actual stuff. And the actual stuff (software/websites) is most of the times so general on the one hand, but very specific to the customer, i don't see how these could be patented. I read that the way a website operates can be patented in the USA. Back to the patent '449. In the press release found on i4i's website, they claim that Microsoft infringes claims 14,18 and 20 from the patent. Those claims refer to the ability of using a lookup-table, being able to generate it, being able to read and replace the values, and being able to use more than one. That is so general! They could sue anyone who is using xml and some kind of look-up mechanism. I think the patent should just be declared invalid. But i guess that is easier said than done. Speaking about the seperation between content and representation, speaking of metacodes, reminded me of Donald Knuth, inventor of TeX, and author of the greatest series of books of all time: The Art of Computer Programming. So let me point you to a letter by that same man, claiming mathematical ideas or algorithms should not be patented. Final thoughts: i4i LP is a company that guards the patents from i4i. So this could be some kind of patent troll? Secondly, they choose Texas, where a large number of these kind of cases have been won, because of their technological inaptitude. I am looking forward to Microsoft's next step. What are your thoughts?

Spatial DB Advisor

PL/SQL package: lessons learned

Working with Oracle Spatial a lot, I found a great source of information in the Spatial DB Advisor, which besides a lot of interesting articles, also offers his source-code and PL/SQL packages for free. The site is not very user-friendly (trouble of seeing the threes through the wood), as finding the free packages is something that i only succeed in coming from Google :) A shortcut: you can find them here. But a word of warning: if you try to install this package in an existing schema, it will delete all indexes and tables. This is something i had to discover the hard way (painful i might add). So i have adapted my version of create_test_data.sql, where at the top of the file all indexes and tables of the user are dropped. Nevertheless, extremely useful package. I will list my favourites (for now):

  • isCompound: checks whether a geometry contains any circular/arc elements
  • ConvertGeometry: converts any special elements - circulararcs, rectangles, circles - in a geometry to vertex to vertex linestrings
  • to_2d: converts a sdo_geometry to 2d. Very useful for us, as geoserver can't draw 2d, and we only use 2,5d anyway. Meaning that we always use a topview, and the z is the elevation level of the ground-level. Or for pipelines: below ground :)
  • sdo_mbr: function to determine the minimum bounding rectangle! i will use this to adapt my script to fill user_sdo_geom_metadata. Cool :)
  • functions to investigate the data of sdo_geometry (number of rings, number of vertexes, number of coordinates, the coordinates itself, ...)
  • functions to investigate the meta-data
  • functions to manipulate your sdo-geometry: scale, rotate, affine transformations, move, adding and removing points, ...
  • ... and lots more ... In short: awesome!! So after getting a terrifying scare, now rebuilding the databases (and adapted the script: won't happen to me again). I mailed the original creator to add a note of warning on his site too. Might help other absent-minded developers :) I also had another smaller problem, because the install-script overruled my ORACLE_HOME and PATH definition, which made sure SQL*PLUS was not found. But that was easily cured. I am very grateful that he puts up this package for free. And i learned another great lesson today: not to get too enthusiastic before i am sure that it all works ;)
Windows 7 and XP

I have been trying the Windows 7 beta, and i must admit it is pretty awesome. What is less awesome is the fact that now the RC is released, and i have to reinstall all my Windows 7 machines (three --i admit i was a bit too enthusiastic). And if i do that, once the final version is released, i have to re-install them again. For my standard day to day use not too bad, but not for my development machine. So i am back to Windows XP now. With a great new look: Zune XP Theme. Nice :) Working with a lot of open source products lately, so it would be nice to use an open source windows alternative. For the moment it would still be very hard for me to just not work on Windows. I still have to develop for .NET and MicroStation and AutoCAD, all only working on Windows platform. So I hope that ReactOSwill become more mature soon. It is really improving fast lately, but still not completely ready to replace Windows. I will have a test-drive real soon. What are the other options? Run Linux and do my Windows work inside a VM? Buy a Mac and do the same? Any suggestions?

When storing spatial data into Oracle, there are a few steps one needs to complete (as shown here too):

  • insert the data (obviously)
  • update the USER_SDO_GEOM_METADATA table. This table specifies for each column the bounding box and the SRID (coordinate system).
  • create a spatial index I have created a script to this automatically for me, once all tables are filled with their data (e.g. after import or after using FME to import your data). [sourcecode language="sql"] set serverout on DECLARE schema_orig varchar2(100) := upper('&user_orig'); CURSOR ctab is select TABLE_NAME from sys.dba_tables where owner = schema_orig; tn sys.dba_tab_columns.TABLE_NAME%TYPE ; CURSOR ctabcol is select column_name,DATA_TYPE from sys.dba_tab_columns where owner = schema_orig and table_name = tn ; collist varchar2(2000) ; query varchar2(2000) ; column_name varchar2(200); has_geometric_column boolean; col_count number; BEGIN -- dbms_output.enable(1000000); FOR ctabrec IN ctab LOOP tn := ctabrec.table_name ; has_geometric_column := false; col_count := 0; for ctabcolrec IN ctabcol LOOP if ctabcolrec.data_type = 'SDO_GEOMETRY' then has_geometric_column := true; col_count := col_count + 1; column_name := ctabcolrec.column_name; -- insert data into user_sdo_geom query := 'INSERT INTO USER_SDO_GEOM_METADATA (TABLE_NAME, COLUMN_NAME, DIMINFO, SRID) VALUES ('''||tn|| ''' , '''|| COLUMN_NAME ||''', MDSYS.SDO_DIM_ARRAY( MDSYS.SDO_DIM_ELEMENT(''X'', 0, 10000000, 0.000005),MDSYS.SDO_DIM_ELEMENT(''Y'', 0, 10000000, 0.000005), MDSYS.SDO_DIM_ELEMENT(''Z'', 0,10000000, 0.000005)), &srid)'; dbms_output.put_line(query); BEGIN execute immediate query; EXCEPTION WHEN OTHERS THEN dbms_output.put_line('**** failed to insert row into user_sdo_geom_metadata for '||tn||'('||column_name||')'); --rollback; ignore any errors! there will be existing columns END; -- create spatial index query := 'CREATE INDEX '||tn||'_'||to_char(col_count)||'_SX ON '||tn||'('|| COLUMN_NAME ||') INDEXTYPE IS MDSYS.SPATIAL_INDEX'; dbms_output.put_line(query); BEGIN execute immediate query; EXCEPTION WHEN OTHERS THEN dbms_output.put_line('**** failed to create spatial index for '||tn||'('||column_name||')'); --rollback; ignore any errors END; end if; END LOOP; END LOOP; commit; END; / [/sourcecode] The script has two parameters: the schema from where to scan all tables, and the srid (coordinate system) which needs to be filled in. Any thoughts on this? suggestions?

I am now working in Ruby on Rails for about a month. I have done a few smaller things before, investigating, trying out; but now I am really building a full application site in Rails! It feels great :) But of course, just starting out in such a new environment, both ruby and rails, I encountered a lot of initial problems. One of them is showing international, accented characters, like é, è, ê ... from the database. Nor Rails nor irb could show the correct characters. I really needed to solve this problem, so this started an investigative journey.

The symptoms

When displaying a table of results from Rails in any browser, all accented characters were shown as white question marks inside a black rhombus (diamond?). When i tested this with irb i also got very weird results. When using TOAD the results of a query are displayed correctly. But maybe TOAD just does it very clever? If i use the GUI version of SQL*Plus (sqlplusw) the results are also shown correctly. But when i use the console (command-line) version of sqlplus, the characters were also distorted. So maybe the clue could be found there.

Oracle NLS-LANG

No matter what character-set the Oracle database uses to store the data (e.g. UTF-8, UTF-16) the data is always converted to the clients character set. This is truly an amazing feature. This depends on a setting called NLS_LANG. Not sure wat it quite stands for. NLS_LANG is built up as follows:NLS_LANG=language_territory.charsetThe NLS_LANG property is set by default in the registry to the correct Windows codepage.NLS_LANG="AMERICAN_AMERICA.WE8MSWIN1252"But why doesn't it work inside the command line console? Apparently because it used a different codepage. If you type chcp (requesting the codepage) at the DOS prompt, it would normally return 437 (it did on my machine). So you would need to enter`

set NLS_LANG=american_america.US8PC437 require 'oci8' OCI8.new('user','pw','db').exec('select * from emp') do |r| puts r.join('|'); end `in your irb and then all results would be displayed correctly. The crucial line being the correct setting of NLS_LANG. Wow! I got my results correctly in ruby! Now i was in the assumption that Rails would be a piece of cake, but that was wrong.

Fixing Rails

The easy idea would be to set NLS_LANG in Rails correct, before the oci8-library is required. My first approach was to set the NLS_LANG in the first line of the environment.rb with the following line:ENV["NLS_LANG"] = "AMERICAN_AMERICA.WE8MSWIN1252"But this didn't work. I am using NetBeans 6.5, and it took me a while to realise that if I edited a file to contain special characters (fixed text, e.g. on a menu) it would work. NetBeans (or Rails for that matter) standard works with UTF-8. So all files are encoded in that way. The easy solution would be to useENV["NLS_LANG"] = "AMERICAN_AMERICA.AL32UTF8"But that didn't work. I tried the alternative AMERICAN_AMERICA.UTF8 but that didn't work either. I am just guessing here, but i think somehow the NLS_LANG setting didn't get picked up in the Rails environment. It kept using the registry setting. So I tried to turn it around: make Rails use the windows codepage instead of utf-8. This took several steps:

  • change the default setting inside NetBeans to use the correct code-page instead of UTF-8. This in fact only affects the format of the files that are saved, but it is nevertheless important that all files being served are in the same format. Also convert all previously edited files from UTF-8 to standard ASCII. I used an editor to do that.

  • added the following to application_controller.rb:` before_filter :headers_iso

    def headers_iso

    make sure the charset matches the default Oracle NLS setting

    headers["content-type"]= "text/html; charset=windows-1252" end `

  • added the correct META-tag to application.rhtml in the HEAD-section:``Note: this meta-tag is not really needed, it has no real effect. And that finally worked! :)

Last night WinSock XP Fix saved my life

I have this machine at home, running XP, and all of a sudden it is getting incredibly slow at startup. I installed Ad-Aware to make sure i don't have any hidden spamware running. It found nothing. So then i tried to find ways to make XP run or boot faster again. I know that the really failsafe way is re-installing my XP but i don't really have the guts to do that, reinstall all drivers and programs. So i tried my friend, Google. I found several articles allowing me to tune my xp-performance and startup time, i applied the ones i found sensible, the ones that were mentioned accross different pages. In short i think the best things you can do is uninstall unneeded programs, make sure that unwanted programs in the background can't start (msconfig). Bootvis seems interesting too, not entirely sure if it helped a lot. Defrag of course! :) Making sure your pagefile can't get defragged by making it large enough, preferable on a separate disk from your boot-drive. These changes seemed to improve my boot, although it is no where near a quick 20 seconds now :) It still takes a few minutes to get my computer completely ready. But suddenly i also lost my network connection. I was puzzled, did i stop one service too many? :) Everything seemed to work fine, but my network-card did not look for an IP address (DHCP). I tried to start my Firewall and i got an error stating the WSAStartup had failed to start, or was not started. WSAStartup? What is that? Looking on the internet i found a small tool, fixing WinSock problems, possibly related to de-installing programs. You never know. And i had no other clues to follow anymore. The tool is called WinsockXPFix. It supposedly could also backup my registry, but that failed in my case. So i just started it, in blind faith, knowing i had to re-install anyway if it wouldn't work. But guess what: i did! The program ran, rebooted automatically, and everything was back in working order! Windows XP, at the time, was promoted for its boot performance, it's self-tuning capabilities, yet over time it only becomes slower and slower. How does Vista handles this? Or Windows 7? Or is it inherent to Windows OS, and should we look at other alternatives. Sometimes i hate how much is done behind the scenes, how much is added. Each vendor or software installs its own version of "Automatic Updates", slowing down my system. Should I look at Mac OS X? Or Linux? Yet Linux still feels to "hardcore" for me, but maybe i am wrong. Should we look at different open source alternatives (in order of viability):

  • Visopsys? this seems like a single developer's playground. Not quite sure what kind of future this holds. But what a great effort to complete alone, though :)
  • Syllable? not quite sure about this one either, either a Linux-based server, or a desktop which is based on Amiga :) That is very nice, i sometimes long for the old times with all the different operating systems out there: QDOS (Sinclair QL), AmigaDOS, Acorn Archimedes, Next. But reviving something like that will never be able to compete with anything commercial. Although i know Mac OS X is heavily based on Next, and rightly so. Next was anyway far ahead of its time. Not to mention they looked nice. I have a soft spot for matte black :)
  • SkyOs: this seems promising, seems to be based on BEOS, which was very promising at the time. It has a lot of applications (open source) available, and a concept of a Software Store: a central place to find all software. A bit like AppStore maybe, and sounds like a good idea.
  • Linux, it all its different flavors
  • Wine allowing to run Windows-application under Linux, makes Linux even more interesting :)
  • Hackintosh: running Max OS X on cheap intel computers. Sounds like a viable option too, albeit a little illegal, since Mac OS X is only allowed to run on Apple hardware. That is a shame though. Would be tempted to try that, just to know for sure Mac OS X is my thing before i buy such a machine.
  • ReactOS: i have no idea how good this is at the moment, but this sounds like an extremely interesting idea to me. Create an open source, improved, version of Windows XP. You can run all current Windows applications on it. I guess they hope to stay up to date. Looks like a very interesting option. So i am also not quite sure what the impact should be, changing OS. I guess application-wise it doesn't matter too much nowadays. There is Open Office, AbiWord, .. which allow you to most of the day to day jobs. Not sure if i could miss PhotoShop though. There is GIMP, and GIMPShop should make switching a lot easier. And such a new OS could revive my older hardware. Maybe.
Patience

It has been the strangest year for me, professionally. I had a good job, where i was hired to do C++ in a Windows CE environment, but where i ended up doing the back-end in SQL Server and C#/.NET. Where i also learned about WPF and WCF and was really impressed. WCF is really easy. Dead-simple. But I really love WPF! It feels like HTML/CSS for Windows UI, except more powerful :) Granted, it took me a while to really get to grips with it, it is not as easy as WinForms is. Although it is not complicated, it's just a different way of thinking. Then i got another offer, from a company where i worked before, long ago, as a youngster, before i became self-employed. I was not really looking, but it felt welcome. I was reluctant somehow, because the other programmers in the company would be leaving, but they promised that would give me the chance to make a new start. I told them i was not going to be able to do it all alone, and needed support. That would not be a problem, they said. So back to geographical information systems. It sounded awesome! I saw the product they were selling, and we could improve that easily. It was web-based, so that would be my chance to finally get into that. I was looking at all kinds of technologies, thinking of .NET, SilverLight, Ruby and RoR. When i arrived there was one guy, not willing to share any info with me. I had to maintain and keep 5 projects up and running on my own, while i didn't even know them from scratch. Communication, like in with a client, is not really my cup op tea. I think i am good at explaining a program, sometimes i think i would like teaching, but i am not good at giving commercial info. All i could think off: sorry, i am all alone here, so i think you will need to wait a few months before i can start on your project. Which i assumed would not sound good, but it was the truth, so i kept silent. Which i believe, in all honesty, is even worse. I told our so-called commercial guy that we needed to communicate to all our clients they were going to have to be delayed, i needed to be worked in, i needed time, ... But all he did was trying to push me to take care of his pet-project. The other projects he just didn't care about (or at least: didn't seemed to). At the same time, there were also a lot of requests for changes to the software used inside our own company. And i was looking around at the possible future implementations. New technologies. It was a though, hard, and stressy time. Too many balls to keep in the air, and juggling never was my strong point. Now, we are almost 5 months further. I have implemented software in VBA for MicroStation and AutoCad. If they had told me beforehand most of my implementation work would be VBA, i might reconsider :) But let's not get into that. I have written one application to calibrate data along a pipeline, in C# and WPF, and show it graphically. That was awesome. We hired a junior, which I try to coach. Not very good i am afraid, with all the work pressure. But coming January i will have a new boss (project manager/commercial) and two more new collegues shortly after. The ambition of my new boss is simple: become the market leader in what we are doing now. Great! That sounds awesome. The last year i have been looking around at so many new technologies. I have played a bit with ruby. I tried out Ruby on Rails. I want to get my hands dirty. And soon! But we need to soothe our clients first, which means: improving their existing systems, before we continue on and create something new. Has anybody ever heard of WebDev? Well i hope you haven't, but i need to work in it. It should be extremely powerful, but maybe i just don't see that yet. Patience is a virtue.

Hiring a new developer

At the company where i am working now, i need to hire new developers/programmers. Remember, the IT-department of this company is constituted entirely by me alone. Problem is: how to recognize a good developer? Luckily there are some very useful articles available on the net:

  • Joel's Guerilla-Guide to Interviewing
  • Interview questions for Candidate Software developers (can you see where he got his inspiration from :))
  • Write a String Reverser (and use recursion!) It has been a long while for me to interview people. I don't want it centered around a specific technology, i just want to be able to grasp, somehow, whether or not they are good developers. Good developers, which i consider myself to be, will get their head round a new programming language in no time. I feel these interviewing hints (mentioned above) will help me finding the right collegue. On top of that, the environment I am working in is still a bit volatile, we could still choose for any valid technology at the moment (java/j2ee, C#.NET, RoR). I guess it will depend on my new collegues experience (and mine). I am favouring either .NET (because i know it well, because it is very developer friendly) or RoR (new, but very promising). If i get a very good java collegue, i am willing to go down that road too.
ORA-01031 error when creating database

At my new job I am back to administering Oracle. The last time I had to do any dba-job, they were still using Oracle 7 :) At my current job, the IT-department can be summed up in one word: me! So, now i have to do everything again. So today, i was trying to get a database installed on a laptop left behind, with no passwords. I was able to solve that, luckily. I had a script to create the database in Oracle 8, had to port it to 9. The biggest change there was: instead of svrmgrl you have to use sqlplus. So i could run the script, but kept getting ORA-01031 error (insufficient privileges). At first: didn't use a sys-account, then lost password of sys-account (set it again using Enterprise Manager), logon using sys-account, but my network user had no ora_dba rights. Logon with a local user with adminstrator rights, put him in the right group, and then it still didn't work ... And at those times, I am so happy there exists this something called internet. I found my solution here. I would never have thought of that. Go figure: a whole day spent trying to create a database. MS SQL Server or MySQL seem so much easier. Industry leader or not, you would suppose they could make equally simple. Or maybe i just haven't found it yet. Don't mention import and export. That is also a pain. But once it is runnning ... :)

Removing VBA Password in AutoCAD

Both AutoCad and MicroStation can now be automated/extended using VBA. This is fabulous. In my current working environment, these programs are used extensively, and a lot of software has been written for it. But unfortunately as well, a lot of programmers have come and went again. So i was left with a program to change, which was locked using a password. A VBA password. I am not quite sure why one would do such a thing, one valid reason would be that some user would accidently get into to the code, and change stuff? No idea. Stop reverse engineering? I could get into a whole discussion about protecting knowledge and sharing knowledge. Anyway, i needed to get in the code, and the person was either unreachable or not willing or forgot the password himself. So i looked for ways to break a VBA password. Apparently there is a very simple way, which took my hours to find. In the *.dvb file, look for the tag "DPB=", and replace that with "DPx=" using a hex editor. Load the project, click through all error-messages, open the properties of the project, and set the password again to something you know and remember, and after that you can remove the password protection all together (if you want). A more detailed description can be found here. This method seems to work for Excel, AutoCAD, let me know if it works for other VBA password protected documents or applications (e.g. Word/Access/... ??).

Firefox 3 beta 5: switch off autocomplete

I find the new Firefox 3 beta 5 unbelievable. It is much faster than the older version. But one of the new "features" bothers me a lot. I don't like the new address-bar autocomplete function. The location bar is now very clever, and if you start typing a word, it will look for all similar words and tags in your bookmarks and history. This is all swell if you don't know the exact url where you want to go, but most of the time i do. Then this option becomes counter-intuitive. And frankly, i don't like how it looks. It is too big. Too much in my face. Maybe i still need to get used to it. But there is an alternative. There are two ways in which the behaviour of the urlbar can be configured. If you type "about:config" in your location bar, you can configure Firefox 3 beta 5. If you then look for the "browser.urlbar" there are two relevant settings:

  • matchOnlyTyped: set this to true, and you get more or less the same behaviour as before, it will only match on url, but it will still sport the new look
  • maxRichResults: if you set this to zero, you disable the location bar autocomplete function entirely. So it is up to you, which you prefer. Mind you, if privacy is the issue, like embarassing url's popping up while typing an url (who likes to admit an unhealthy Disney addiction), record-keeping in Firefox can now easily be disabled. Just open Extra->Options->Privacy and unselect the "keep history for at least ..." checkbox.
Easy XML reading using XmlTextReader

When looking at examples and samples for the .NET System.Xml.XmlTextReader, i was hoping it would be as easy to use as XmlTextWriter. But i did not find such examples. XmlTextReader will read node per node, using the method Read, and a programmer will have to evaluate what type of node the current node is. But XmlTextReader has two short-hand methods: ReadStartElement and ReadElementString. Suppose we have the following xml : [sourcecode lang="xml"] <root> <itemlist> <item> <name>Item1</name> <value>110</value> </item> <item> <name>Item2</name> <value>234</value> </item> <item> <name>Another Item</name> <value>432</value> </item> </itemlist> </root> [/sourcecode] What we want to do is read one item in a simple way. That is possible as follows: [sourcecode lang="csharp"] private void ReadItem(XmlTextReader reader, out String name, out String value) { reader.ReadStartElement("item"); name = reader.ReadElementString("name") ; value = reader.ReadElementString("value"); reader.ReadEndElement(); //item } [/sourcecode] Unfortunately ReadStartElement will throw an exception if the current element is not the same as the wanted element. So we need a bit more code to iterate smartly over all possibilities. [csharp] private void ReadItemList(XmlTextReader reader) { reader.ReadStartElement("itemlist"); reader.Read(); // read next element while (reader.LocalName == "item") { String name; String value; ReadItem(reader, out name, out value) ; // ... do something here with read item ... reader.Read(); } reader.ReadEndElement(); // itemlist } [/csharp] The reader.Read() positions ourself on the next node, allowing some kind of look-ahead, while the ReadEndElement and ReadStartElement can handle this perfectly well. They will either proceed to the next node, or stay on the current node. That's why this code works. For completeness, this does the setup: [csharp] XmlTextReader importReader = new XmlTextReader(fileName); importReader.ReadStartElement("root"); ReadItemList(importReader); importReader.ReadEndElement(); [/csharp] ... where filename points to the file containing the above shown xml. If you have any questions or comments regarding this example, please let me know. This allows for very readable and concise code, but only if you know how the xml file you are reading is formed (the order of the elements). I will solve this in the next post.

WCF maximum string length

WCF and MaxStringContentLength on XmlDictionaryReaderQuotas

When sending xml-data over WCF using a string, i bumped into the following error :

The formatter threw an exception while trying to deserialize the message: Error in deserializing body of request message for operation 'Test'. The maximum string content length quota (8192) has been exceeded while reading XML data. This quota may be increased by changing the MaxStringContentLength property on the XmlDictionaryReaderQuotas object used when creating the XML reader.

Right. This sounded like some foreign language where i can't even make out the syllables. I am using WCF with a NetTcpBinding, so after extensive googling i was able to produce the following C# code : [sourcecode lang="csharp"] ushort myport= 27001; const Int32 myMaxMessageSize = 6553500; string address = String.Format("net.tcp://localhost:9000/myservice/{0}", myport); XmlDictionaryReaderQuotas quotas = new XmlDictionaryReaderQuotas(); quotas.MaxStringContentLength = myMaxMessageSize; NetTcpBinding binding = new NetTcpBinding(); binding.ReaderQuotas = quotas; binding.MaxReceivedMessageSize = myMaxMessageSize; binding.MaxBufferSize = myMaxMessageSize; m_factory = new ChannelFactory<imyservicecontract>(binding); m_proxy = m_factory.CreateChannel(new EndpointAddress(address)); [/sourcecode] Setting the MaxStringContentLength alone doesn't cut it. Once your buffer grows larger than 65K (the default for MaxBufferSize and MaxReceivedMessageSize ) you will run into errors again. So that's why i opted to change those as well. Hope this helps :)

Well apparently MS SQL Server 2005 does not accept remote connections by default. Luckily i was not the first to bump into this problem, here is a step-by-step explanation to solve that. Kudos to those guys! :)

Nieuw begin

Momenteel ben ik o.a. actief in de e-learning sector, waar ik ook vooral zelf veel moet bijleren. In zo'n zoektocht naar nieuwe tools en informatie, heb ik via de blog van de CEO van Intalio "Getting Things Done" (GTD) ontdekt. Dat lijkt bij nader onderzoek nogal hip te zijn. Ongelooflijk veel tools en blogs die het over GTD hebben. Weer een bos en de bomen probleem :) Maar momenteel ben ik dus volop bezig mijn eigen werkwijze te stroomlijnen en te formaliseren, en ik zou het zelfs ruimer durven stellen : mijn levenswijze. Een pracht-site in dit verband is zenhabits.com. Alles om je leven zo zen mogelijk te krijgen. Huishoudtips, geld-tips, gtd-tips, alles vind je erop :) Maar binnenkort meer hier over productiviteit, en alle nieuwe tools.

Beyond Java (Bruce Tate)

Good thing about having to change, is thinking about your career, where you want to be, where you want to go to. I am a c++ man. By stagnation. I have done a bit of java, but not enough to my liking to call me proficient. The language shouldn't pose a problem, but the frameworks do. So i picked up this book. The Java environment is getting bigger and bigger, and coming from the outside, the step towards it is also harder and harder. Not because of the language or the complexity of the frameworks, but just the sheer size and the amount of available solutions, options. I wanted to look "Beyond Java", aim for the next wave directly. The author focuses on web-programming. He feels this is the way to go. I agree, for all client-based software. GUI would best be written for the web, so you don't have deployment problems (or less), scaleability from the start (or easier, just scale your webserver --or it should be that easy). So this book focuses on two solutions : Ruby On Rails and Continuation servers. Both seem to make web programming easier. I have used PHP and MySQL, which is easy but dirty. Ruby on Rails seems much neater, a cool environment, the scaffolding is a good start. RoR as an environment, for pure webprogramming, beats java hands down. Continuation servers are really cool, because they allow the programmer to think like for a normal UI-application. Much easier. But the downside is that for now it is only available on SmallTalk. Ruby by the way, as a language, looks really interesting. It is great. Reminds me a lot of SmallTalk too. But ... how cool RoR looks, it is limited. Java is a total enterprise framework, which is way bigger then just programming for the web. Web-programming in Java could be improved (port Rails ?), but Java offers more. Integration with legacy is possible. Java is also the platform where all the latest developments come together naturally. Refactoring, design patterns, agile development, was all nicely integrated. This makes java, in my book, a winner. There is as we speak no real contender. What about .NET ? Well, i have seen it at work. It is super-cool. The development environment is awesome. C# has some really cool stuff, and does do things better than java. Building forms is dead-easy, and they look good. I have heard building websites is as easy (or easier). For the websites i am not as convinced because the strength of webbrowsers has to be the platform-independency. So you don't include components which can only be used or shown on a Windows PC, or worse, on a Windows PC with the entire .NET environment present. But on that regard : i am not an expert, so i am not sure what .NET is capable of. For me it just doesn't feel as right, working with .NET which totally ties you to a single supplier. Porting applications to linux becomes entirely impossible. Should you want to ? Well, for instance, our application servers ran 30% quicker on less performant Intel machines. Go figure. For me there seem to be two options. Learn Ruby, and fast. Learn java, even better. Ruby for fun, java for real work ;-)

About generics and stuff

In my new job, which will start on the 1st of February, i will first work in c++, and then be moving on to .NET. This was not entirely what i had in mind. I was planning on moving to java, completely. On such short term though, this seemed difficult, because up until now i haven't had much *real* experience with java. Although, pure java programming, coming from c++ is very straightforward. But i am unfamiliar with the standard libraries. Even worse, making the leap bigger, i am definitely unfamiliar with the J2EE framework, and all possible frameworks available to make your life easier (struts/spring/hibernate..). When switching to java, the problem no longer is learning the programming language, but getting to know all the available frameworks. And it is a big leap. As a former Sinclair QL adept, i would say Quantum Leap ;-) But the advantage of java is that is more open source. Almost everything is freely available, or you can find good "free" alternatives. There are no ties to specific platforms, or suppliers. That is exactly what scares me about .NET. But looking at it in more detail, it is actually pretty neat. It is designed, mind you, by the guy from Borland, Anders Hejlsberg. My god, Borland ! I am a Borland fan from the start, precisely because of this guy, and because of their war against Microsoft. Well war is not exactly the right word, but i always had the impression Borland was the underdog, with a superior product. Speaking of VCL, Delphi or C++ Builder : it always was a better language than VB, and a great RAD-tool. Visual C++ had its strengths though. Definitely. It was a very impressive compiler, but not RAD. Microsoft made this weird decision that VB would be the RAD-part, and Visual C++ would be for real programmers. For instance, making a make-file in Visual C++ was and still is way better than in Borland C++ Builder. In Visual C++ you can keep different targets, which is very handy. By the way, does anybody remember what RAD stands for ? Rapid Application Development. Nowadays it would be called agile ;-) Anyway, i was looking at Java 1.5, with generics, which i was looking forward to, as it is one of the strongpoints in c++, imho. Anyway, in java it is not at all implemented well. An ArrayList still remains an ArrayList on the inside, because Sun didn't want to adapt the JVM. So all translations are done in bytecode. All casts are inserted. An ArrayList will be converted to Objects, so you lose a possible efficiency advantage. Reflection will not work, because from runtime on, it is just a normal ArrayList. Imo this also means you can still insert other types into an ArrayList. Doh!

In c#, generics are done well, but, alas, different than in c++. At least an ArrayList can only contain integers. That's a relief. But in c# the typing is much stronger than in c++. And I am not quite sure whether this is better or worse. In an age where everyone is pointing at Ruby, and its weak typing, and it's apparent advantages, c# goes to the extremely strong typing. Let me explain. In c++, templates are a kind of macro, which get instantiated at compile-time with a type . This means, that in the template you could use any function, and only at compile-time, will it check whether such a function is available. Whereas in c#, the generic is a real class, and you have to define the constraints on the type , or the class wont compile. Meaning you have to specify the interfaces T implements, for instance, or which classes it inherits from. Which is ok, it is much stricter, it is correct, and it will catch bugs much earlier. Whereas in c++ you could get this weird, unreadable, lengthy errors, when instantiating a template, and some function suddenly was missing. A classic are the comparison operators (well, at least in my experience). It is a design decision of course. I am not quite sure which exactly is the best. Introducing a possibility of weak typing does sound appealing, but also has disadvantages.

As in Ruby, where typing is very weak : if it walks like a chicken, and talks like a chicken, it must be a chicken. In Ruby, all checking is at runtime, it is never type-based, but signature-based. So if a specific class has a certain expected function, doesn't matter which type it has, or where it derives from, but if the function is available and the signature matches, call it. Great. This ultimately is very dynamic, and possibly very strong. It also invites, of course, as always, possibly bad programming. And you will have to use unit-tests very thoroughly to catch all errors soon enough. But it sounds great.

I am going to try out Ruby (and RoR of course), and I will keep you posted.

Clarification (about procrastination)

In my local newspaper I read an article about laziness. Being lazy at work, because your employer wants to make a profit out of you, anyway. And you should do the same, be the first. I might be the exception, i think i am not, but i actually like to work hard, to do the job as good as i can. Procrastinating has nothing to do with unwillingness to work. It is just sometimes, i feel blocked, creatively, which somehow pushes me toward procrastination. I am not lazy, i like to work, i take pride in what i do, i want to earn my living earnestly and honestly. In the article there were 10 rules to follow as an employee. How to be lazy succesfully. In short : lay low. Don't be noticed. One rule i would like to mention though :

Rule 9 : always be nice to contractors, on a fixed-time contract, because they are the only ones who really work.

Aaaaah, yes ! At last, recognition :)

Future

Weird how things go. On my private website, i have written a review on the book "A whack on the side of the head", by Roger Von Oech. Which is a great book, about creativity, about how a whack to the head might get you started, and how you could whack your creativity, without hurting yourself. I have been whacked, recently. My contract has been ended. Which was a bit of a surprise. But it also got me started again. Got me thinking about my carreer, how i would want to progress. How i would go about accomplishing this. Learning again. And then suddenly, opportunity comes along, with a totally new and different road. And I like it. I hope it works out ! Wish me luck ;-)

Oracle 10 Troubles --continued

We had more than a few troubles migrating to Oracle 10, in our production environment. We are using a clustered server, with 3 nodes, so it should be blindingly fast. I'll sum up our biggest issues (we had) :

  1. Sequences : we used a sequence to make sure the messages we send, can be ordered. Somehow our sequence was set to "NO ORDER", but in a single-node environment this always works. In our environment, we suddenly go gaps, leaps, depending on which node you were connected to. This seemingly unexplainable behaviour (sequences are always in order, so the error must be something else), was luckily quickly discovered (are they really?), and easily fixed.
  2. ORA-01483 : all our c++ server software, running on NT, is built using C++ Builder and BDE. We have had troubles like hell with "max open cursor" problems. So, fix, or workaround : count maximum open cursors, if this approaches the database maximum, re-connect to the database. Dirty, but it works. So, in Oracle 8, we had upped the SESSION_CACHED_CURSORS to 250. When we did the same in the Oracle 10g, suddenly, out of the blue appeared the ORA-01483 error. Whammo !! It took us quite a while to figure out what exactly caused the errors. Resetting the aforementioned to it's default value fixed that problem.
  3. Database drop-out : every night our database just crashes, hangs. This is related to "open cursors problem". Our applications count the amount of open cursors using the v$-tables, which apparently after a while causes Oracle to hang. Here is the query which correctly counts the open cursors given the program name : [sql] select distinct a.value, s.sid, s.machine, s.program, s.inst_id, s.server from gv$sesstat a, gv$statname b, gv$session s where a.statistic# = b.statistic# and b.name = 'opened cursors current' and s.sid = a.sid and s.inst_id = a.inst_id and upper(s.program) like '[program-name]%' order by 1 desc [/sql] The query we used before, was : [sql]select count(*) from v$open_cursor where sid=:sid[/sql] To solve this, i changed the server-process causing the database to hang each night, to use DOAinstead of BDE. Apparently this also alleviates our open cursor problem. I did notice that open cursors, using DOA, are only really closed in Oracle after a commit. This is taken into production tonight, let's hope it will finally improve our databases up-time. For the moment we manually stop and start our database twice a day, in a controlled way, in the hope our database can keep up for the rest of the time. Oracle are looking into our case, i am not quite sure why our setup is different than the rest of the world. It has been a very hectic two weeks. Finally we are reaching a somewhat stable environment, although it still requires a lot of manual intervention to keep it running smoothly.