Updates

Here's an update on my recent works. Our story left off at enhancing the page allocator and pager code that I've written for the SharpOS kernel. Since then I've spent my time (what time I have between other obligations) doing a number of things:

Extracting ADC from PageAllocator

I've started to work out the logistics for an ADC module. Many things remain unclear. It's obvious that in an ideal situation we'd want each ADC layer to be placed in individual DLLs which can be AOT-compiled-in when we build the kernel. This allows us to only have to build the kernel once for all platforms supported with ADC (:-D). However, at this point in the game the AOT can't do multi-DLL compilation (but I think only a small change is required). Right now I've got two ADCs, SharpOS.ADC.X86 and SharpOS.ADC.Generic. I've posted X86 in my sandbox folder, but the usefulness of Generic (which is really just a 'null' implementation) is shaky.

AOT branch

I also started my own AOT branch which has some nice trivial changes for things like option support and output verbosity. The most notable thing I did was write a few scripts and embed them as resources so the AOT would be able to generate a DOS floppy image for testing, directly after compilation (and automatic :-D). This should speed up the testing process a bit, as creating the image files for testing is a real pain on both Windows and Linux.

Enhancing the Page Allocator

Very much aligned with the fact that the control bitmap technique was wholly unsatisfactory, last night I changed the page allocator I'm working on to use the free page stack technique. I even wrote the code to allocate a range of contiguous physical pages. I still must finish paging and the basic memory mapping API.

Building the Memory Manager

I've started work on the memory manager implementation for SharpOS. Mostly this is because it hasn't been done yet and I will need the functionality to test any of the other code (SharpOS.EIC and SharpOS.Runtime) using the SharpOS AOT compiler.

Using various internet resources I have hacked together what is a mega-basic page allocator. It uses the provided Multiboot information to determine the amount of RAM, and the offset/size of the kernel must be passed to MemoryManager.Init(). Using this information, it sets up a bitmap for control information on the first page past the end of the kernel, and reserves the pages below 0x100000 as well as the pages occupied by the kernel. The only thing left to implement for a basic page allocator is the Alloc/Dealloc, which is a matter of binary search.

The allocator also handles setting up 2-level paging and enabling it, but right now it does not relocate anything--so physical == virtual. I'd like to put the kernel in the higher half, but that requires modifying the segment base pointer so that we don't have to relocate pointers (would be tedious). I think I need the AOT's help to manipulate the segment base pointer at the right time (compile-time?).

Of course, the bitmap is not an ideal method of memory management. The bitmap search will be slow once lots of memory is allocated, and I'm looking into various ways of improving the performance. One method I've seen is to use another control bitmap for 'superpages' which represent large blocks of pages. Another method is to use a stack of free pages and just pop and push pages to it for allocation. This is a much faster way of allocating, but apparently gets difficult when allocating contiguous pages. I believe it is the method used by Linux and Windows NT though, which lends credence to the concept.

More later!

SharpOS Foundation

I am currently working on a foundation layer for the SharpOS kernel in the form of a runtime and utility library. The utility library is going to turn out kind of like a C library for C#, providing a String object, an Array object, various linked lists, etc. The runtime has now merged with SharpGC in terms of in what assembly it is (SharpOS.Runtime.dll), and the delegate use has been squelched because I don't think we're going to have an easy time converting delegates to function pointers and back again. Instead, the GC takes a reference to a Runtime* object when it is created and it calls methods on that runtime when it needs to do object-walking and runtime finalization. Oh, I've also moved the Heap* object to the SharpOS.Memory library along with the MemoryManager prototype. This makes more sense in terms of what the Heap does and the namespace it's been occuping (SharpOS.Memory as opposed to SharpOS.Runtime).

I am now working on tests for the EIC objects, then I will look to expanding the set to a few more utilities, and perhaps integrating bmarkham's Math object. Lots to do in SharpOS land.

Sypheon 2: Object-oriented

I once wrote a piece of software called Sypheon, which I used to manage the websites I created. Sypheon is a powerful XML content system that is based on an XML format that describes a page as a set of "blocks" which are arbitrarily placed into an XHTML "template".

When I originally wrote it, PHP's object oriented features were premature and limited, so I wrote it as a procedural application, making little use of modern OO structures. I decided to rewrite it recently as an object-oriented PHP application, which would also let me clean up the architecture a bit (procedural PHP is *really* hard to keep clean and tidy).

Another thing I wanted to address was the frequent use of "dynamic tags" in the older version. "Dynamic tags" are a way to allow both data blocks from pages and templates to insert dynamic information. This was the main way to express to Sypheon that a certain block should be inserted at a certain point in a template. The syntax, however, was extremely limited. Also, due to the way the files were processed, normal PHP tags (IE, < ?php someFunc(); ?>) could not be used. Instead, there was a 'php' dynamic tag which would be used like this:

Special tricks were employed to allow the : character to be used in the PHP code, which was reserved as the argument delimiter for dynamic tags.

In the new Sypheon, everything is based on PHP itself, so instead of using dynamic tags one would use just the < ?php ?> stuff. A special class is exported (called XCS) before the preprocessing of pages and templates which interfaces with Sypheon to output certain information. So the call in the template to show the "title" block at a certain point would be:

Much cleaner. Well, today Sypheon2 generated the first correct page, which is a Hello World of sorts, and the code is furiously smaller than the first incarnation. Next up is implementing the editing and page caching features of the old version. This time it will be easy. And elegant.

Signing off!

Connection Reset

Does anyone remember the days before broadband? Today I got a random Connection Reset error on some website, which was quickly fixed by reloading the page. The phrase "connection reset" quickly sparked a path of synapses somewhere, causing me to remember my days with only dialup. You could go with a local service that connected at 56k, but had a download speed of 3.3kb/s (because the ISP cheaped out) or a national service, the most popular of which being AOL. There were other national services, but most didn't have any decent access numbers for many users. So your choice was lower price and local business, or double the speed, shaky connectivity, evil. Oh, and a huge world of advertising, free to access by proprietary keyword. Bah.

Yes, the times when loading pages almost always took more than 5 seconds (gasp! no no, not again!) instead of the comfortable 2-5 second span we broadband users typically enjoy (nonscientific data-collection methods implied here). A time when downloading any sort of file was a long and ardous process, from the largest (maybe a game demo or media) to the low shelf of around 500kb, which, assuming a (top notch for dialup i might add) transfer rate of 10 kb/s, would take 10 seconds.

Eh, jusht 10 shecondsh?

I won't get into the plethora of reasons that 10 seconds can be ardous and unhealthy for you (oh, especially around the time that broadband became prominent among other people you knew), but if you took the evil option (that's AOL, for the snoring), you can also look forward to turning 10 seconds into a continuous gamble. A dramatic horror movie climax, if you will, where the intensity grows as the progress bar stretches toward it's goal. Quickly, youthful databytes! Race quickly before the road falls apart a(Connection reset.)

...gain.

SharpOS GC

I'm currently working on a garbage collector written in C# for the SharpOS project. It's an interesting project as the code must avoid making use of new managed objects to keep track of the old ones. Without managed objects I'm left with two possibilities--deal directly with unmanaged memory space and pointers or I can have the runtime have all objects explicitly implement an interface which includes the size of the GC tracking data in the size to be allocated. So the GC indirectly allocates the resources necessary to track it implicitly. This work is dependent on having a VMM to handle paging, swapping, etc. Chriss said he looked into it and will handle writing it. The GC code I have currently implements all the appropriate System.GC methods/members which are useful.

I'm following a site that lists the basic compacting/generational GC design. I'll write more as I make the code more capable.