The physical reorganization is going nicely. The additional of 2 new shelving units has allowed us to (finally) organize all the equipment into one location. Now that my back is finally starting to recover it is time to update the logical structure.
Right now, we have several systems running virtual machines. We have an older ESXi server that runs a few older development systems. We also have a subversion server running on a Linux VM, and a mail server running under Hyper-V.
Essentially, we have four systems running different hypervisors: ESXi 3.5, VMware Workstation, Hyper-V and VMware Server 1.0.
This will be changing next week. A new project is forcing us to expand our testing capabilities, and the Dell Outlet had a very nice server at a good price!
We picked up a dual processor T710 with 48GB of Ram to run VMware vSphere 4.1; the Essentials version is only $611. We’ll put a few TB of storage on it, and be able to migrate all the existing VM’s, as well as all the new ones to it.
Our existing ESXi server will be transitioning to the backup server role. I’ve been wanting to try out Openfiler, and this will give me a chance. After another purchase at Newegg, we have an external enclosure, and 12 TB of RAID storage for backup data. We’ll use external drives to archive the data.
This project will allow us to repurpose (or retire) three existing servers and add 10 new “testing” VMs, which will be available all the time! It also allows us to consolidate our backups to a single backup server.
We’re still unsure what third party backup tool to use with vSphere. There are many options, with differing price points. We’re going to start with Veeam FastSCP, since it’s free (and what we are using now).
Over the next several months we will use the 30 day free trials to test the following tools:
- Veeam Backup & Replication
- Quest vRanger
- PHD Virtual Backup
Right now I’m leaning towards Veeam due to it’s ability to restore single files from a VM. To be honest, if it works well enough, I won’t even test the other tools.
November 12th, 2010
With a little help from Junk Busters Austin the office clean up is complete. It was a bittersweet day; getting rid of old CRTs, Pentium-2 servers, and boxes of old software.
The logical cleanup started as well. I’ve set up a new Domain Server using openSUSE 11.3 with LDAP. This was my first experience with openSUSE, and now that it is done I can recommend it.
Major thanks to Masim Suguanto and his tutorial which was a great help with the initial setup: http://vavai.net/2010/03/30/how-to-samba-pdcopenldap-on-opensusesles-part-1/.
Also, it was good to read up on the Samba group mapping http://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/groupmapping.html before starting.
But, getting Windows Server 2008 and 2008 R2 on the Domain was a bit of work!
How to get Server 2008 R2 to operate with a Samba 3.5 Domain Server:
1) Update the registry on the Server 2008 System: http://wiki.samba.org/index.php/Windows7
2) Ensure that the Windows and Linux have the same date and time (run “net time \\Server /set” on the Windows Server).
3) Modify the Server 2008 local policy’s for Network Security: LAN Manager authentication level to “Send LN & NTLM – use NTLM2 session security if negotiated”.
4) Modify the Server 2008 local for Network Security: Network security: Allow Local System to use computer identity for NTLM (Disabled): http://wiki.ssystems.de/doku.php?id=samba_trust_w2008r2_harald_strack.
5) I also set the AllowNT4Crypto registry setting, though I’m not sure if that was needed: http://technet.microsoft.com/en-us/library/cc974327%28WS.10%29.aspx
Reading this, it sure seems easy now! Though, it took me the better part of two days to get the Samba Domain Controller, LDAP, and Server 2008 all working together. I like the new database style configuration used by openSUSE 11.3, as well as the GUI configuration tools.
Migrating the local profiles to domain profiles was not as bad as I expected. I ran the “User Profile Wizard 3.0” and it seemed to work nicely.
The Next Step:
With the new domain server in place, my next step will be to migrate all my utility and test VM’s from an ESXi (3.5) Server, VMware Server for Linux, and VMware Workstation to a new Dell Server. I’ve gotten very tired of having all the VMs scattered among several different systems, and want to consolidate them in one place.
Once it is running, the old ESXi server is going to become the new backup server (openfiler seems to be a good choice). Well, once I plug in a new eSATA card to allow me to hook up all the external drives I’ve been using, that is.
October 25th, 2010
A few days ago an update of qt.gitorious.org added the Visual Studio 2010 mkspec. This means that I can finally use scripting and webkit with the vc10 compiler.
Previously, I had been building Qt with the vc9 (2008) mkspec, and -no-webkit -no-script -no-scripttools.
To pad this post a bit, here’s a link to another blog post where somebody is using Qt, F#, and .dot/mono: qt-from-fsharp. I have to admit that I *like* building GUIs using XAML, but for cross-platform applications Qt is the way to go.
May 2nd, 2010
Monday morning I downloaded and set up the new Visual Studio on my main development system. As a C++ developer, the new features and fixes are pushing me to migrate active projects to it.
I’m excited to use the new C++ features. Auto, lambda, and static_assert are my current favorites.
The What’s New in Visual C++ 2010 has a list of all the new language goodies.
Qt doesn’t have a mkspec yet. I believe that 4.7 will officially add VC 10 to the supported list, and remove VC 7.1 (sadly, since I think it was the best compiler Microsoft has released).
Last night our deployment server died. It had been running (fairly) problem free for over 6 years. We used it to host our internal FTP, Bug Tracking and Wiki software.
The silver lining is that we can migrate to a hosted service. This would save us from needing to buy a new Server. More importantly, we wouldn’t have to take care of it, manage software updates or back it up.
This would also let us move a few projects to a hosted svn server. Our external developers would appreciate the speed boost.
I’m leaning towards using xp-dev.
April 14th, 2010
My plans for a well organized office have been thwarted by a rush project targeted at Window CE. I’ve never written anything for CE before, and have learned quite a bit about it over the last month.
First, CE isn’t the same as Windows Mobile — CE may be customized quite a bit for the device it is targeted at. This makes it a little challenging to build the software; especially when the client isn’t sure what the hardware folks are doing.
Second, the “recommended” tools and SDK are extremely old (based on VC 6). Luckily, you can use the newer tools (VC 2008) as long as you’re careful about your build settings and the System API’s you are using.
Lastly, it’s amazing how dependant I’ve become on the FPU. This project was targeted at an ARM CPU. These don’t typically have FPUs (though it is an option). Most of the rendering code I write uses floating point.
It’s not that ARM processors are slow, they are rather quick and I appreciate their shifting ability – it’s that floating point is emulated with a software library. This makes it imposable to use in time sensitive areas.
It reminds me of writing for the 80386.
So, after benchmarking the rendering code and discovering just how slow the math was, it was time to implement a fixed point library. Years ago, when Intel was still developing their XScale, they had an optimized fixed point library for the ARM. This seems to be long gone from the Internet – I looked for it for several hours and sent several emails to Marvell without any luck.
As it turns out, my ‘C’ fixed scale code was reasonably fast, and the VC2008 compiler did a decent job of optimizing it. After tuning the routines to prevent over and underflows the module performed quite well.
The only magic is to use 64 bit integers to multiply and divide – if you don’t have those, it’s a lot tougher.
The result was a fast software renderer, with optional support for DirectDraw and GAPI blit’ing. It was a lot of fun to get into “old school” software mode.
April 3rd, 2010
Over the past few months I’ve been working on several confidential projects. They have prevented me from spending much time on *new* technology. Though, one of the projects has involved a new Database, and has been interesting.
I have been thinking about my goals for the year.
- I will complete porting my main codebase to Visual Studio 2010.
- I will complete my F# tour.
- I will completely clean my office.
It isn’t that my office is a huge mess, rather, it has a lot of older items stored in it. For example, I can count 6 blank VHS tapes sitting on a shelf. Getting ride of all the old items, and reorganizing the equipment that I actually use would allow me to gain a huge amount of space.
With that complete, I would be able to buy a few metal racks and finally put all the systems in one place. Last year, I “found” two operating routers when I was moving a printer. I’m not sure what bothered me more; that I had forgotten about them or that they were buried so deeply under cables that they were not visible.
My goal is to have everything on racks, visible, with cables zip tied neatly.
I was enjoying my progress with F#, and can’t wait to get back into it. The immutability coupled with threading excites me a great deal.
The Visual Studio 2010 release seems to have corrected the flaws introduced with the 2005 and 2008 releases. My C++ benchmarking has shown it to be almost as fast as Visual Studio 2003, and a bit faster when PGO was used. Once the mobile SDKs are available, I may be able to migrate all my active projects to it.
March 7th, 2010
Over the weekend I installed the Visual Studio 2010 Beta 2 and the Silverlight 4 Beta. On my laptop, it installed fine. On an older (spare) development box it caused Windows 7 to “black screen” in the middle of the install process. I admit that this is an older system, and may be a bit unstable, but it shouldn’t crater on install.
My two goals are to:
(1) Try the new features of Silverlight 4; printing and privileged applications. These were the key issues that prevented me from hosting applications in Silverlight.
(2) Begin the process of porting a large project from Visual Studio 2003. VC 2005 and 2008 tried to “fix” the MS runtime libraries via Windows SxS. In my mind, the result was far worse! VC 2010 goes back to the older model, where the application will load the DLL’s in the application directory (no fuss, no muss).
Because I’m in a C++ mood, I’m starting with the port. The first step will be to build Boost – the trunk has support (for 2010), so it’ll just launch bjam and let it churn. Qt doesn’t have a mkspec for 2010 yet, so I’ll just use the 2008 settings.
Assuming nothing goes wrong, I’ll start the basic porting this afternoon.
December 2nd, 2009
My adventures with Silverlight have encouraged me to get up to speed with C# and .Net 3.5. I’m dividing my time between Windows and Mono (and Moonlight) under Linux.
I’m reading “Pro C# 2008 and the .NET 3.5 Platform”, by Andrew Troelsen (ISBN-13: 978-1-59059-884-9), and can give it a good recommendation.
For my first project, I have decided to implement a Hilbert R-tree and see how well it compares with the overlapping Quad-Trees that we have now.
R-trees use nested dynamic rectangles to store information. Because they are created based on the data which is stored in them, the structures should provide better utilization, that is no empty (wasted) nodes.
The difficulty with R-trees is that it is difficult to decide how to split data into child rectangles. The goal is to put an equal number of objects into each of the new child rectangles and minimize overlap.
The Hilbert R-tree solves this problem by using a 2D Hilbert curve to index the data. A Hilbert curve is a continuous fractal space-filling curve (wiki) which will visit all points of a k-dimensional grid without overlapping.
By partitioning the data using the Hilbert number of it’s center point, we can easily split a node into several child nodes.
The Hilbert R-tree also balances itself via node reinsertion (like the R* tree).
In any case, this seemed like an interesting task to use for exploring C#.
I started by porting my existing overlapping quad tree to C#. Then I implemented a Hilbert R-tree, without the reinsertion logic, to see how well the splitting logic worked.
Testing Results
Using random data, with a limited size (to better emulate typical mapping data), the quad tree was about 4x faster for inserts.
The R-tree had about %25 fewer nodes.
Search time for the quad tree was slightly better. I believe that this is due to the large amount of overlapping of the child nodes in the R-tree.
I modified the R-tree splitting to partition the data based on the largest distances between the Hilbert values, which increased the node count, but increased the search speed to roughly the same as the quad tree.
Next Step
The operation of spatial trees is incredibly dependant upon the data. My tests are using randomly generated data, which is interesting, but ultimately useless.
Next, I’ll extract several hundred thousand objects from a dataset, and use them for performance testing.
October 13th, 2009
I’ve been reading up on Silverlight quite a bit. So far, one book has has impressed me: “Data-Driven Services with Silverlight 2”, by John Papa. It has filled in some gaps.
I can also recommend “Pro Silverlight 2 in C# 2008”, by Matthew MacDonald. It covers the major parts of the client side.
September 11th, 2009
I continue to experiment with the Silverlight Virtual Earth control, as well as the MapCruncher utility.
The Silverlight control does a reasonable job of displaying raster data. I enhanced my sample to show a birds’ eye view, and allow pushpins to be added to the map.
Sample
I used the MapCruncher to try and construct a mosaic from a set of raster data which I generated. Don’t misunderstand, for a Beta Product, it did a reasonable job with a handful of raster images, but when I tried to run a batch job with several thousand – not so good.
Part of the issue is that my test data is in an interesting projection. I have been using it to test my rendering pipeline, and never actually received the projection data. In any case, I fudged a projection function that looked reasonable.
After five attempts to run batches of images, I took a step back and just rendered a single huge PNG file for each map layer.
The results are pretty good.
The primary negative is that the MapCruncher output is huge! My source data was 70MB. The MapCruncher data size was 850MB.
I would consider using MapCruncher for small amount of data, but nothing large.
For bigger raster datasets, I would consider using the ERDAS Image Web Server. I spotted a blog entry which has some asp.net files that would allow IWS to handle Virtual Earth requests.
For my vector and polygon data, I would create a web service to generate raster data on demand for dynamic map layers. This would be fun, since I could change the display properties of the different objects types and simply re-render the map tile.
Conclusion
The Silverlight Virtual Earth control shows a great deal of promise for consumer, and light weight applications. I was easily able to generate custom raster base maps and load vector data via a web service.
Being forced to use the Mercator projection limits its usefulness for more traditional applications.
August 15th, 2009
Next Posts
Previous Posts
