Operating system and programming language

[RedX]

I understand that you can write a complete operating system in a high-level language like C++, compile it into machine code, and then write it to storage, and then sell it or give it away.

But is this historically what was done? Did the compiler come first, or the operating system?

When you download a compiler today, it's for a specific operating system (because you need to be able to type in your program into a text file, open a program to translate it into machine code, and instruct your hardware how to write your program to storage - all of which require the operating system). This seems to suggest the operating system came first.

But it could have been the case that the first computer was programmed in machine code to only have the basics: input/output, a compiler, and a way to write to storage. Once this computer is made, an operating system could be created by using the high-level language compiler and writing it to storage.

Also, what is the relationship between libraries and the operating system? I heard that both Mac and PC have the same processor (x-86 chip), but they don't run the same programs. But surely the same 0s and 1s on the same processor should give the same results? I heard the reason has to do with libraries, but don't libraries become obsolete once you have the 0s and 1s of the program? Once the 0s and 1s are loaded in your CPU, then nothing else matters.

 

[mgb_phys]

But is this historically what was done? Did the compiler come first, or the operating system?

You can use an existing operating system to bootstrap a new one.
Windows before NT essentially used DOS to load windows, the earliest versions of Linux used minix and a number of experimental OSs today use Linux.
Eventually you can write the lowest level drivers and boot loaders and the OS becomes self hosting.
A similar problem arises with a new CPU, here you use cross compilation, you compile on machine A running one OS using a special version of the compiler that emits the machine code for machine B's CPU

When you download a compiler today, it's for a specific operating system

Compilers like GCC, where you have the source code be built on lots of different OSs, they can even do a bootstrap mode where a minimal GCC is ported to a new OS by hand, this compiles an intermediate GCC which then gives you the support to build the full GCC.
You can also make GCC output code for another OS, it's possible to build windows apps under Linux with gcc.

But it could have been the case that the first computer was programmed in machine code to only have the basics: input/output, a compiler, and a way to write to storage. Once this computer is made, an operating system could be created by using the high-level language compiler and writing it to storage.

Yes, although you could also use an existing open source OS to provide all those in the early stages.

Also, what is the relationship between libraries and the operating system? I heard that both Mac and PC have the same processor (x-86 chip), but they don't run the same programs

crrect.

But surely the same 0s and 1s on the same processor should give the same results? I heard the reason has to do with libraries, but don't libraries become obsolete once you have the 0s and 1s of the program?

The libraries are still used to talk to the rest of them machine, so the set of commands to draw on the screen in windows is different to mac.
You can get the same library on multiple OSs which talk to the underlying system library for you. So for instanceif you program in Qt or OpenGL the command to draw a line is the same (as far as you are concerned) but the Qt library will have different calls inside it on Windows and Mac.

Once the 0s and 1s are loaded in your CPU, then nothing else matters.

That's true of anything inside the CPU that doesn't talk to the rest of the machine. A sin() calcualtion would be the same pattern of 0/1s. But the layout of functions in an executable migth be different so a Mac wouldn't be able to run a windows program.

 

[lostinxlation]

I understand that you can write a complete operating system in a high-level language like C++, compile it into machine code, and then write it to storage, and then sell it or give it away.

But is this historically what was done? Did the compiler come first, or the operating system?

Old computers didn't have very complicated features. They didn't have virtual memory, they didn't have multitasking, they ran all the instructions in sequential mode and all the instructions were synced before executing the next one. With such a system, you can just load the program written in the machine languages, load the data, set up the program counter and other registers from the IOs, trigger it from IO, and it runs as written in the program, with or without operating system. In fact, you can do the same with today's computers, but it was certainly much easier to do it back then.
I don't know which one came first(though most probably OS came first). but technically, whichever way worked easily in the early day of computing.

 

[mgb_phys]

Sorry I thought you meant for a new machine.
Historically compilers came first - the first compiler was for Fortran in 1953

Machines didn't really have an OS until the early 60s, before then they had the simplest driver to load a program (such as a compiler),run it and output the result (either the result of the program, or in the case of the compiler another program)

Even today you don't need a OS. Some simple embedded computers in your washing machine or TV remote just run a single program and sit in a loop waiting for an input and writing an output.

 

[waht]

Also, an operating system is a wall between a program and hardware. When a program, for example in C++, is compiled, the result is actually a complicated and tedious list of system calls. When the program is executed, its the system calls that ask the OS to allocate memory, and processor time, or talk to hardware. A library is basically a Rosetta Stone from a high level language to system calls.

There are generally two major architectures of this, one is a micro-kernel which is windows. Windows can allow programs to directly talk to hardware. The second one is a monolithic like unix, linux or mac, and those completely 100% separate programs from directly talking to hardware.

That's why a program complied on windows won't work on mac.

A cool command line in ubuntu is strace. It can show you all the system calls of any executed program.

 

[RedX]

Also, an operating system is a wall between a program and hardware. When a program, for example in C++, is compiled, the result is actually a complicated and tedious list of system calls. When the program is executed, its the system calls that ask the OS to allocate memory, and processor time, or talk to hardware. A library is basically a Rosetta Stone from a high level language to system calls.

There are generally two major architectures of this, one is a micro-kernel which is windows. Windows can allow programs to directly talk to hardware. The second one is a monolithic like unix, linux or mac, and those completely 100% separate programs from directly talking to hardware.

That's why a program complied on windows won't work on mac.

A cool command line in ubuntu is strace. It can show you all the system calls of any executed program.

I think I have a fundamental misunderstanding of what a compiler is. I thought it converted your high-level language into 1s and 0s for your particular processor, and the job of the operating system is to just load the program into CPU memory. But evidently this is not the case. Instead a compiler converts your higher-level language to OS-dependent system calls. If this is the case, then to me, it seems the operating system itself is a high-level language, and the maker of a C++ compiler needs to familiarize themselves not with a particular processor, but the operating system, to convert C++ commands into OS-commands (system calls). So it seems to me then that operating systems are more fundamental than compilers.

Well maybe it's a hybrid: a maker of a compiler needs to know the CPU machine code and the system calls code of the operating system (as someone noted above, sin(x) is the same on all x86s regardless of operating system).

So taking the example of C++, if you don't load any libraries at all, and just do something like:

void main()
{int x=2;int y=3;int z; z=x+y;}

then after compiling, this program will run on all x86s regardless of operating system, since there are no system calls here, but only 0s and 1s? Actually, if the operating system takes care of memory storage, then even this program is not just 0s and 1s, but also system calls to tell your OS to assign memory locations?

But if you include a library like iostream, then this is definitely operating system dependent? I used to think that whoever wrote iostream did it in machine code. But I guess iostream is written in system calls code that has been specified by the OS manufacturer? So someone has to write iostream for Macs, Windows, and Linux and not just the architecture, since iostream deals with hardware drivers which are taken care of by the operating system?

But it seems to me that if you're a programmer, and are unsatisfied with every single one of the OS's system calls, then instead of using the OS's system calls, you can just write that part of the code in the machine code of the processor instead of system calls code - but only on Windows, since Macs and Linux don't allow you as a programmer control of hardware devices in machine code, but only through the prescribed system calls?

 

[rcgldr]

Without cpu and hardware specific extensions to a compiler, the part of an OS that deals with hardware can't be written in C++. For example, I/O ports on a x86 type cpu are not memory mapped, so you need a port I/O function to communicate with these. Special cpu instructions to work with interrupts, do task switches, ..., also can't be implemented in C++, so some assembly is required.

As far as which came first, compiler or OS, it depends on what you're willing to call an OS. If this includes the early OS's that could just load and execute programs, then the OS came first.

The first instances of programming were "hard wiring" vacuum tube computers to run a specific program. The next step was the ability to enter and store programs, and run them from the "data memory" of the early computers.

http://en.wikipedia.org/wiki/ENIAC

 

[waht]

An operating system is basically one giant scheduler on steroids. And so, any program that runs under an OS needs to communicate with it.

Here is an example of a helloworld program in C++ I compiled using gcc, and ran on ubuntu.

#include <iostream>

using namespace std;

int main()
{
	cout << "Hello World\n";
	return 0;
}

Looks simple right? But when executed and tracked all system calls using strace, this how the program interacts with OS just to produce a simple output. Here is a small sample. I've uploaded the text file because it's like 150 lines long.

execve("./helloworld", ["./helloworld"], [/* 36 vars */]) = 0
brk(0)                                  = 0x8fba000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb809b000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=50808, ...}) = 0
mmap2(NULL, 50808, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb808e000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/usr/lib/libstdc++.so.6", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0000B\4

and here are the last few lines:

munmap(0xb808e000, 50808)               = 0
fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 1), ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb809a000
write(1, "[B]Hello World[/B]\n"..., 12Hello World
)        = 12
exit_group(0)

 

[waht]

I think I have a fundamental misunderstanding of what a compiler is. I thought it converted your high-level language into 1s and 0s for your particular processor, and the job of the operating system is to just load the program into CPU memory.

Everything ultimately exists as 0s and 1s. It's just that there is many abstraction layers. And so, in a sense, the compiler converts 0s and 1s into 0s and 1s. But it also interjects system calls, or interrupt requests in order for the program to communicate with the OS.

it seems the operating system itself is a high-level language, and the maker of a C++ compiler needs to familiarize themselves not with a particular processor, but the operating system, to convert C++ commands into OS-commands (system calls).

Yes. It's just a language to interact with the OS. You couldn't code a sin(x) function with it. Most programs don't get to see the actual hardware. For example, an OS allocates virtual memory in pages of certain length.. So if a program requests lots of memory, the OS will allocate it in pages. The program will see a nice continuous memory address, but in reality each page may exist in different parts of real hardware memory - which is managed by the OS.

But if you include a library like iostream, then this is definitely operating system dependent? I used to think that whoever wrote iostream did it in machine code. But I guess iostream is written in system calls code that has been specified by the OS manufacturer? So someone has to write iostream for Macs, Windows, and Linux and not just the architecture,

yes

since iostream deals with hardware drivers which are taken care of by the operating system?

yes indeed, the program won't touch hardware in most cases. It needs to talk to the OS as it takes care of all the hardware.

I'm probably referring to linux and unix here. Windows has more leeway in this matter.

 

[mgb_phys]

I think I have a fundamental misunderstanding of what a compiler is. I thought it converted your high-level language into 1s and 0s for your particular processor, and the job of the operating system is to just load the program into CPU memory

Almost - the compiler does convert the program into 0/1, it also did the same with the system libraries.
The system libraries are there to translate the 0/1s into the correct memory address to talk to a particular piece of hardware for instance.

An operating system does more than just load the program, early 'operating systems' like DOS do just load the program. Then your program has to make the raw memory calls to talk to the hardware - this means your program needs to know things like screen type and disc layout.
An operating system insulates you from lots of this

 

[robheus]

Just as a remark, but the question you raise is exactly equal to the (evolutionary) chicken-and-egg problem. You need an operating system to run a compiler, and you need a compiler to make an operating system.
The answer is that both derived from simpler systems, that functioned interdepently and were evolved in conjunction with each other, and in a similar way, that was what happened to the famous "chicken-and-egg" problem.

 

[RedX]

Thanks to everyone for explaining things to me. I know what I say makes little sense, so that it's hard to figure out where I'm going wrong and respond.

I have some more questions:

As far as I can tell, system calls are libraries in C. So if you are writing a program in say Fortran, would you have access to the system calls?

Is the library of system calls referred to as an API? Or is an API something else entirely?

Consider this hypothetical: Say I download a C compiler that runs on my Windows machine and outputs machine code to an x-86 architecture. If I download the C system library for a Mac, then in my program I can use Mac system calls. But once I compile to get 0s and 1s, can I run the resulting executable on an x-86 Linux machine? I guess what's confusing me is that I learned about the fetch/execute cycle of a processor, and from this picture the same 0s and 1s for the same processor should produce the same output. The system calls are C libraries that are used to write the programs, but once you compile and get an executable, then when the operating system loads the executable onto the CPU memory and gets out of the way, then the behavior should be the same?

The makers of compilers must translate the high level language into machine code, so in order to do this they must spend time familiarizing themselves with the chip architecture of the machine. But they must also know how to convert the system calls provided by the C library into machine code. Are the operating system manufacturers responsible for this?

 

[PaulS1950]

Machine code is processor specific. You would have to run the Mac code with a mac processor or run it through an emulator (whichis very slow).
Once the code is compiled it becomes a language that is only compatible with certain processors. Intel 80X86, Motorola 68XXX and so on. If you use ANSI coding then you can compile the same program on different processors to get the same program that will run on different machines (more or less).

 

[mgb_phys]

Once the code is compiled it becomes a language that is only compatible with certain processors

That was the OP's confusion, a MAc and a PC use the same processor - so the opcode for eg. multiply is the same - but you can't take a PC program and run it on a mac.

If you use ANSI coding then you can compile the same program on different processors to get the same program that will run on different machines (more or less).

You still need OS specific ansi/posix libraries to implement the actual std lib routines.
And a recompile is still needed to handle different executable layout and different linker standards on the different operating systems.

 

[RedX]

That was the OP's confusion, a MAc and a PC use the same processor - so the opcode for eg. multiply is the same - but you can't take a PC program and run it on a mac.

You still need OS specific ansi/posix libraries to implement the actual std lib routines.
And a recompile is still needed to handle different executable layout and different linker standards on the different operating systems.

Would it be correct to say that a compiler is processor-specific, but a linker is what really
has an operating system dependence?

I thought a compiler only makes 0s and 1s, but I read that although the .o files have the 0s and 1s, the memory addresses associated with variables are still unresolved. A linker is what assigns a virtual memory address for each variable, and the OS can translate a virtual memory address to physical memory address through a translation table.

So the picture of a program being just 0s and 1s to be loaded on the processor is really naive (or rather really old because that's how computers were in the old days). The program needs to be in the right executable format for the OS, which is a combination of 0s and 1s that are processor specific but also virtual memory addresses which are OS specific?

 

[mgb_phys]

Would it be correct to say that a compiler is processor-specific, but a linker is what really has an operating system dependence?

At some level that's a reasonable picture.

So the picture of a program being just 0s and 1s to be loaded on the processor is really naive (or rather really old because that's how computers were in the old days). The program needs to be in the right executable format for the OS, which is a combination of 0s and 1s that are processor specific but also virtual memory addresses which are OS specific?

A program is still 0/1 s but there needs to be more to say that this is a program and how to run it rather than say a jpeg image - which is also just 0/1s