How to Program a Computer Operating System

Operating systems allow people to interact with computer hardware; they're made out of hundreds of thousands of lines of code. They are usually made with C#, C, C++, and assembly. Operating systems allow you to navigate through a computer while creating storage and executing commands. Programming your own operating system gives you the opportunity to learn a huge amount about computers an programming and allows you to create a system that does what you want. It isn't easy to program one and requires a lot of knowledge.

Decide what programming language to use.

While it is possible to create an operating system in a language such as Pascal or BASIC, you will be better off using C or Assembly. Assembly is absolutely necessary, as some vital parts of an operating system require it. C++, on the other hand, contains keywords that need another fully-built OS to run.

In order to compile an operating system from C or C++ code, you will, of course, be using one compiler or another. You should, therefore, read the user guide/manuals/documentation for your chosen C/C++ compiler, whether it comes packaged with the software or is available on the distributor's website.

You will need to know many intricate things about your compiler and, for C++ development, you should know about the compiler's mangling scheme and its ABI. You are expected to understand the various executable formats (ELF, PE, COFF, plain binary, etc.), and understand that the Windows proprietary format, PE (.exe), has been copyrighted.

Decide what media you want to load your OS on. It can be a CD drive, DVD drive, flash drive, a hard disk, a floppy disk, or another PC.

Decide on your core idea of an OS. In Windows, for instance, an easy-to-use GUI and plenty of security is the core idea.

Target what processor platform your operating system will support. IA-32, ARM, and x86_64 are the most common for personal computers so they're your best bet.

Decide if you'll do it all yourself, or use an existing kernel. Decide if you would rather do it all yourself from the ground up, or if there is an existing kernel you would like to build on top of. Linux from scratch is a project for those that would like to build their own Linux distro, for example.

Decide if you'll use your own boot-loader or a an existing boot-loader. Decide if you're going to use your own boot-loader or a pre-created one such as Grand Unified Bootloader (GRUB). Since coding your own bootloader will require extensive knowledge of the hardware and the BIOS, it may push back the schedule for programming of the actual kernel.

Decide on your application programming interface (API). One good API to choose is POSIX, since it is well documented. All Unix systems have at least partial support for POSIX, so it would be trivial to port Unix programs to your OS.

Decide on your design. There are monolithic kernels and micro kernels. Monolithic kernels implement all the services in the kernel, while microkernels have a small kernel combined with user daemons implementing services. In general, monolithic kernels are faster, but microkernels have better fault isolation and reliability.

If you want an easy way, consider Linux distros like Fedora Revisor, Custom Nimble X, Puppy Remaster, PCLinuxOS Mk LiveCD, or SUSE Studio and SUSE KIWI. However, the operating system you create belongs to the company who started the service first (though you have rights to distribute it freely, change it and run it however you like under the GPL).


Start small.

Begin with small things such as displaying text and interrupts before moving on to things such as memory management and multitasking. You can also try making a simple 16-bit Operating System, instead of taking a big leap.

Consider testing your new operating system with a virtual machine.

Rather than rebooting your computer each time you make changes or having to transfer the files from your development computer to your test machine, you can use a virtual machine application to run your OS while your current OS is still running. VM applications include VMWare (which also has a freely available server product), the open-source alternative, Bochs, Microsoft Virtual PC (not compatible with Linux), and Oracle VirtualBox.

Check for potential deadlocks and other bugs. Bugs, deadlocks and other problems will affect your Operating System project.

Be sure to implement security features as your top priority if you ever want your system to be viable.

Keep a backup of the last working source.

This provides a measure of protection in case something goes terribly wrong with the current version of your OS or your development. If your computer crashes and is unable to boot, it is an excellent idea to have a second copy to work with so you can troubleshoot. (If you need keep a backup, make sure there are 2 - 3 disk partition. 1 OS as an OS will develop another one, and 1 backup of old version (older, not oldest), and 1 partition for current version).

Release a "release candidate."

This will allow users to tell you about potential problems with your operating system, or the rating about your operating system.


Tips

Do not wipe your hard drive completely.

Remember, wiping your drive will irreversibly clear out all your data! Use GRUB or another boot manager to dual-boot your system with another OS until yours is fully functional. It may be a good choice to make a completely new partition for "developing" the OS.

Carelessly writing your operating system to the hard drive can corrupt it completely. Be careful.


Consider developing and working in a team.

That way, less time is required to solve more problems, which may produce a better OS more quickly.

Use websites such as OSDev and Bona Fide OS Development to help you develop your own operating system.

Please note well that for most purposes, the OSDev.org community would prefer that you stick to using their wiki, and do not join the forum.

If you do decide to join the forum, there are prerequisites: You are required to thoroughly know C or C++, and x86 Assembly language. You must also understand general, and complex programming concepts such as Linked Lists, Queues, etc. The OSDev community, in its rules, explicitly states that the community is not around to babysit new programmers.

Do not start an operating system project in order to begin learning programming. If you don't already know C, C++, Pascal, or some other suitable language inside out, including pointer manipulation, low-level bit manipulation, bit shifting, inline assembly language, etc., you are not ready for operating-system development.

You are also required to have read the processor manuals for the processor architecture you have chosen; whether x86 (Intel), ARM, MIPS, PPC, etc. The manuals for a processor architecture may be easily found using a Google search ("Intel Manuals", "ARM manuals" etc).

Do not join the OSDev.org forums and start asking obvious questions. It will simply result in "Read the Manual" answers. You should try reading Wikipedia, and the manuals for the various tools you wish to use.


Warning

Do not expect that a proper operating system will be easy to build. There are lots of intricate inter dependencies. For example, in order to make an operating system able to handle multiple processors, your Memory Manager must have "locking" mechanisms in place to prevent multiple processors from accessing the same resource at the same time. The "locks" used for this will require the presence of a scheduler to make sure that only one processor accesses a critical resource at any given time and all the others are made to wait. Yet the scheduler depends on the presence of a Memory Manager. This is a case of a deadlocked dependency. There is no standard way to solve problems like this; each operating system programmer is expected to be skilled enough to figure out his own way of dealing with it.