Most modern OSes don't treat the physical RAM as a single contiguous area, from address 00000000 to 1fffffff (say, on a 512MiB machine). Instead, they have hardware (the MMU, or Memory Management Unit) which maps physical RAM to a logical address space. This is used heavily by the operating system to move memory about, usually in 4K chunks called "pages".
On systems with 32-bit address spaces, the total available address space is 4GiB. This space will be used sparsely, with different chunks of physical RAM mapped to different areas of the available logical address space. Each area of logical memory will correspond to, say, the kernel, or kernel data, or an application, or even a chunk of shared memory owned by several applications.
When an application tries to read a piece of data from a (logical) memory address, the MMU hardware looks up the logical<->physical mapping, and routes the request to the right bit of the physical memory. However, if there's no physical memory available corresponding to that logical address, it raises an exception on the CPU, which goes into a "page fault" mode, and executes the code at a given address.
An OS can put its own code at the "page fault handler" address, and deal with the error condition gracefully -- normally by terminating the application in question with a signal. In the case of Unix-derived OSes, this signal is traditionally called a segmentation fault. On Windows, it's a general protection fault.
Note that this page-fault mechanism is also the way that virtual memory ("swap") is implemented -- on a page fault, the OS checks its own look-up table of swapped-out pages. If the logical memory being addressed has been written to the swap area on disk, then it will find a little-used page of physical RAM, write that data to the swap space, remap the physical/logical RAM mappings to move the newly-freed up piece of physical RAM to the right place, and then load the required data for the new page from disk, and return control to the application that triggered the page fault in the first place.
The MMU is also used to protect processes from each other. On Windows 3.1, all processes were mapped into memory at the same time, so a process which went wrong could see and write all over the memory of other processes. In modern OSes (including Linux and Windows NT/2000/XP), memory protection between processes is performed by only having one process (and any shared memory that it uses) mapped into memory at any one time. When the kernel does a context switch from one process to the next in its scheduling queue, it remaps the memory so that the "outgoing" process temporarily no longer has any of its pages in the logical address space.