Rust Runtime

Recap: Last Week
run::Process::new(program, argv, options)
spawn_process_os(prog, args, env, dir, in_fd, …)
fork()
int 0x80

libc: fork()
jumps into kernel code
sets supervisor mode

linux kernel: fork syscall
12 November 2013

University of Virginia cs4414

1
Plan for This Week
• How the Kernel Makes a Process
– Virtual Memory

• Thursday: diving into fork.c

12 November 2013

University of Virginia cs4414

2
From Class 3:

Batch Processing

Program

Computer
Center

Your Program Runs

Output: Invalid Operation
Charge: $174.32
12 November 2013

University of Virginia cs4414

3
Process Abstraction
Provide each program with the illusion that it
owns the whole machine.
The best example of this way to do things
is Linux, which is an operating system,
which is a program that keeps track of
other programs in a computer and gives
each its due in space and time.
Guy Steele, “How to Grow a Language”
HT: Anonymous for posting link in Piazza
forum
12 November 2013

University of Virginia cs4414

4
Memory Isolation
Process 1 should only be able to access Memory Space 1
Process 2 should only be able to access Memory Space 2

Memory
Space 1

12 November 2013

University of Virginia cs4414

Memory
Space 2

5
Software-Based Memory Isolation
Original Code

“Sandboxed” Code

…
movq %rax, -8(%rbp)
…

…
movq -8(%rbp),%rdx
andq %rdx,%rgx
movq %rax, %rdx
…

Safe
Loader

Assumes %rdx is reserved and %rgx
is protected and holds a mask for
the memory segment

12 November 2013

University of Virginia cs4414

6
SOSP 1993
SOSP 1993

12 November 2013

University of Virginia cs4414

7
12 November 2013

University of Virginia cs4414

8
Hardware-Based Memory Isolation
Original Code

Running Code

…
movq %rax, -8(%rbp)
…

…
movq %rax, -8(%rbp)
…

Memory
Space 1

12 November 2013

University of Virginia cs4414

Memory
Space 2

9
Virtual Memory
address in Process P

Virtual Memory Mapping
physical address
owned by Process P

User-level processes cannot access physical memory
directly: all memory addresses created by process P are
virtual addresses, mapped into physical addresses
owned by process P
12 November 2013

University of Virginia cs4414

10
Getting Into the Details…

12 November 2013

University of Virginia cs4414

11
SOSP 1967

Procedure Base Register:
segment number of executing
procedure
Argument Pointer
Base Pointer
Linkage Pointer
Stack Pointer
Descriptor Base Register

12 November 2013

University of Virginia cs4414

12
Generating an Address
18 bits

18 bits

218 = 262144
12 November 2013

University of Virginia cs4414

13
Addressing Mode selects:
Argument Pointer
Base Pointer
Linkage Pointer
Stack Pointer
12 November 2013

University of Virginia cs4414

14
Descriptor Base Register

What does
MULTICS
need to do
to switch
processes?

12 November 2013

University of Virginia cs4414

15
12 November 2013

University of Virginia cs4414

16
1982

12 November 2013

"It used to be that programs were
easy to copy and change. But
manufacturers began to lose money
as many people made copies of
software and gave them to their
friends. Now, many manufacturers
have figured out how to 'copyprotect' discs. A copy-protected disc–
like a cartridge–can’t be copied or
changed. To our mind this is a
disaster: Most people learn
programming by changing programs
to fit their own needs. This capability
of customization is what makes
computers so attractive. New ways of
copy protection will probably be
found soon. Until then, a computer
owner may have to put up with being
'locked out' of his own machine.”
Popular Mechanics, January 1982

University of Virginia cs4414

17
Intel 80186

Intel 80286
First x86 Processor with Virtual Memory Support
“Protected Mode”

12 November 2013

University of Virginia cs4414

18
http://en.wikipedia.org/wiki/File:Intel_i80286_arch.svg
12 November 2013

University of Virginia cs4414

19
Five x86-64 Processor Modes
Real Mode: pretend to be an 8086
20-bit direct-access address space
Protected Mode: “native state”
System Management Mode: platformspecific power management and
security (separate address space)
Compatibility Mode: pretend to be
x86-32
IA-32e/64-bit Mode: run applications “For brevity, the 64-bit submode is referred to as 64-bit
in 64-bit address space
mode in IA-32 architecture.”
12 November 2013

University of Virginia cs4414

20
Protected State (can
only be modified by
the kernel):
RFLAGS (includes EFLAGS)
Includes I/O Privilege Level
Control Registers
CR0 bit 0: controls if processor
is in protected mode
CR3: page directory base
register

12 November 2013

University of Virginia cs4414

21
Address Translation

Memory

University of Virginia cs4414

Paging
Unit

Physical Address

Linear Address

Logical Address
12 November 2013

Segmentation
Unit

22
Accessing Memory

16 bits to select segment, 16- 32- or 64- bits to select offset
Actual addressable space for user-level process in Unix:
247 bytes = 128TiB

12 November 2013

University of Virginia cs4414

23
Memory

Paging
Unit

Physical Address

Linear Address

Logical Address

Segmentatio
n Unit

Computing
the Linear
Address

Segment selection
is inferred from
instruction type

Logical (“General”, “Virtual”) Address
Segment
Selector

12 November 2013

Offset

University of Virginia cs4414

24
Fetching an Instruction
EIP
CS

Instruction Pointer (Offset)

32 bits
16 bits

Code Segment

13 bits

1

Table Index

12 November 2013

University of Virginia cs4414

Ring
Global or
Local Table

Only Kernel can write to Segment Registers

2

25
Segmentation Tables
Global Descriptor Table (GDT)
limit

base address

linear
address
space
0-264 - 1

Segments can overlap!
13 bits – up to 8192 entries
12 November 2013

University of Virginia cs4414

26
Segmentation Tables
Global Descriptor Table (GDT)
Local Descriptor
Table
(per process)

13 bits – up to 8192 entries
12 November 2013

How does the processor find the GDT/LDT?
University of Virginia cs4414

27
The GDT and LDT are just
data structures in memory!
Special registers store their
locations

12 November 2013

University of Virginia cs4414

28
Logical Address

Segmentatio
n Unit

Linear Address

Paging
Unit
Physical Address

Memory
12 November 2013

University of Virginia cs4414

29
Linear Address

Logical Address

64
2

Segmentation
Unit

Paging

linear addresses
What would it cost to have 264 bytes of RAM?

12 November 2013

University of Virginia cs4414

30
$10 per 1GB = 230 bytes

264 bytes = $10 * 234

$172B ~ ½ Google’s Market Cap
12 November 2013

University of Virginia cs4414

31
Paging
Memory

Paging
Unit

Physical Address

Linear Address

Logical Address

Segmentation
Unit

We don’t need to store the whole address space in memory!
Most of it is unused, and we can store rarely-used parts on the disk.
12 November 2013

University of Virginia cs4414

32
Memory

Physical Address

Linear Address

Paging
Unit

Image from Wikipedia
12 November 2013

University of Virginia cs4414

33
Overview (Intel 386)
32-bit linear address
CR3

Dir

Page

10 bits
(1K tables)

Page
Directory

Offset

10 bits
12 bits
(1K entries) (4K pages)

Page Entry

Page Table

Physical
Memory
Page + Offset

CR3+Dir

12 November 2013

University of Virginia cs4414

34
Page Table Entries
CR3
Page Entry

Page Table
Page
Directory

12 November 2013

20 bits: physical
address of page
12 bits: flags
user/kernel page
write permission
present
University of Virginia cs4414

Physical
Memory
Page + Offset

35
386 Checkup
32-bit linear address
CR3

Dir

Page

10 bits
(1K tables)

Page
Directory

Offset

10 bits
12 bits
(1K entries) (4K pages)

20 bits addr / 12 bits flags

Page Table

Physical
Memory
Page + Offset

CR3+Dir

How many pages do we need to store the page table?
12 November 2013

University of Virginia cs4414

36
How slow is this???!
Memory

Page

Paging
Unit

Physical Address

Dir

Linear Address

Logical Address

Segmentation Unit

Offset

CR3

Page
Directory

Page Table

Physical Memory

GDTR

Global
Descriptor
Table
12 November 2013

University of Virginia cs4414

37
Translation Lookaside Buffer (Cache)

Memory

Page

Paging
Unit

Physical Address

Dir

Linear Address

Logical Address

Segmentation Unit

Offset

CR3

Page
Directory

Page Table

Physical Memory

GDTR

Global
Descriptor
Table
12 November 2013

University of Virginia cs4414

38
Page Fault
CR3
Page Entry

Page Table
Page
Directory

12 November 2013

Physical
Memory

20 bits: physical
address of page
12 bits: flags
user/kernel page
write permission
present
University of Virginia cs4414

39
How common are page faults?

12 November 2013

University of Virginia cs4414

40
top -o mem -stats pid,command,cpu,mem,mregion,vsize,faults
12 November 2013

University of Virginia cs4414

41
Physical Memory:
9106M used (2553M “wired” – cannot be paged out)
+ 5090M unused
= 14196M
where is my missing ~2GB???!

Virtual Memory: 594G (total)

top -o mem -stats pid,command,cpu,mem,mregion,vsize,faults
12 November 2013

University of Virginia cs4414

42
How expensive is a page fault?

12 November 2013

University of Virginia cs4414

43
#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv) {
char *s = (char *) malloc (1);
int i= 0;
while (1) {
printf("%d: %xn", i, s[i]);
i += 4;
}
}

12 November 2013

What will this program do?

> ./a.out
0: 0
4: 0
8: 0
12: 0
…1033872: 0
1033876: 0
1033880: 0
1033884: 0
Segmentation fault: 11

University of Virginia cs4414

44
Charge
• Make progress on your projects: everyone
should have a clear idea what you are doing
now
• Will post more details on next deliverable
(design reviews) soon
Challenge: write a program that takes N as an input and
produces (nearly) exactly N page faults.

12 November 2013

University of Virginia cs4414

45

Virtual Memory (Making a Process)

  • 2.
    Rust Runtime Recap: LastWeek run::Process::new(program, argv, options) spawn_process_os(prog, args, env, dir, in_fd, …) fork() int 0x80 libc: fork() jumps into kernel code sets supervisor mode linux kernel: fork syscall 12 November 2013 University of Virginia cs4414 1
  • 3.
    Plan for ThisWeek • How the Kernel Makes a Process – Virtual Memory • Thursday: diving into fork.c 12 November 2013 University of Virginia cs4414 2
  • 4.
    From Class 3: BatchProcessing Program Computer Center Your Program Runs Output: Invalid Operation Charge: $174.32 12 November 2013 University of Virginia cs4414 3
  • 5.
    Process Abstraction Provide eachprogram with the illusion that it owns the whole machine. The best example of this way to do things is Linux, which is an operating system, which is a program that keeps track of other programs in a computer and gives each its due in space and time. Guy Steele, “How to Grow a Language” HT: Anonymous for posting link in Piazza forum 12 November 2013 University of Virginia cs4414 4
  • 6.
    Memory Isolation Process 1should only be able to access Memory Space 1 Process 2 should only be able to access Memory Space 2 Memory Space 1 12 November 2013 University of Virginia cs4414 Memory Space 2 5
  • 7.
    Software-Based Memory Isolation OriginalCode “Sandboxed” Code … movq %rax, -8(%rbp) … … movq -8(%rbp),%rdx andq %rdx,%rgx movq %rax, %rdx … Safe Loader Assumes %rdx is reserved and %rgx is protected and holds a mask for the memory segment 12 November 2013 University of Virginia cs4414 6
  • 8.
    SOSP 1993 SOSP 1993 12November 2013 University of Virginia cs4414 7
  • 9.
    12 November 2013 Universityof Virginia cs4414 8
  • 10.
    Hardware-Based Memory Isolation OriginalCode Running Code … movq %rax, -8(%rbp) … … movq %rax, -8(%rbp) … Memory Space 1 12 November 2013 University of Virginia cs4414 Memory Space 2 9
  • 11.
    Virtual Memory address inProcess P Virtual Memory Mapping physical address owned by Process P User-level processes cannot access physical memory directly: all memory addresses created by process P are virtual addresses, mapped into physical addresses owned by process P 12 November 2013 University of Virginia cs4414 10
  • 12.
    Getting Into theDetails… 12 November 2013 University of Virginia cs4414 11
  • 13.
    SOSP 1967 Procedure BaseRegister: segment number of executing procedure Argument Pointer Base Pointer Linkage Pointer Stack Pointer Descriptor Base Register 12 November 2013 University of Virginia cs4414 12
  • 14.
    Generating an Address 18bits 18 bits 218 = 262144 12 November 2013 University of Virginia cs4414 13
  • 15.
    Addressing Mode selects: ArgumentPointer Base Pointer Linkage Pointer Stack Pointer 12 November 2013 University of Virginia cs4414 14
  • 16.
    Descriptor Base Register Whatdoes MULTICS need to do to switch processes? 12 November 2013 University of Virginia cs4414 15
  • 17.
    12 November 2013 Universityof Virginia cs4414 16
  • 18.
    1982 12 November 2013 "Itused to be that programs were easy to copy and change. But manufacturers began to lose money as many people made copies of software and gave them to their friends. Now, many manufacturers have figured out how to 'copyprotect' discs. A copy-protected disc– like a cartridge–can’t be copied or changed. To our mind this is a disaster: Most people learn programming by changing programs to fit their own needs. This capability of customization is what makes computers so attractive. New ways of copy protection will probably be found soon. Until then, a computer owner may have to put up with being 'locked out' of his own machine.” Popular Mechanics, January 1982 University of Virginia cs4414 17
  • 19.
    Intel 80186 Intel 80286 Firstx86 Processor with Virtual Memory Support “Protected Mode” 12 November 2013 University of Virginia cs4414 18
  • 20.
  • 21.
    Five x86-64 ProcessorModes Real Mode: pretend to be an 8086 20-bit direct-access address space Protected Mode: “native state” System Management Mode: platformspecific power management and security (separate address space) Compatibility Mode: pretend to be x86-32 IA-32e/64-bit Mode: run applications “For brevity, the 64-bit submode is referred to as 64-bit in 64-bit address space mode in IA-32 architecture.” 12 November 2013 University of Virginia cs4414 20
  • 22.
    Protected State (can onlybe modified by the kernel): RFLAGS (includes EFLAGS) Includes I/O Privilege Level Control Registers CR0 bit 0: controls if processor is in protected mode CR3: page directory base register 12 November 2013 University of Virginia cs4414 21
  • 23.
    Address Translation Memory University ofVirginia cs4414 Paging Unit Physical Address Linear Address Logical Address 12 November 2013 Segmentation Unit 22
  • 24.
    Accessing Memory 16 bitsto select segment, 16- 32- or 64- bits to select offset Actual addressable space for user-level process in Unix: 247 bytes = 128TiB 12 November 2013 University of Virginia cs4414 23
  • 25.
    Memory Paging Unit Physical Address Linear Address LogicalAddress Segmentatio n Unit Computing the Linear Address Segment selection is inferred from instruction type Logical (“General”, “Virtual”) Address Segment Selector 12 November 2013 Offset University of Virginia cs4414 24
  • 26.
    Fetching an Instruction EIP CS InstructionPointer (Offset) 32 bits 16 bits Code Segment 13 bits 1 Table Index 12 November 2013 University of Virginia cs4414 Ring Global or Local Table Only Kernel can write to Segment Registers 2 25
  • 27.
    Segmentation Tables Global DescriptorTable (GDT) limit base address linear address space 0-264 - 1 Segments can overlap! 13 bits – up to 8192 entries 12 November 2013 University of Virginia cs4414 26
  • 28.
    Segmentation Tables Global DescriptorTable (GDT) Local Descriptor Table (per process) 13 bits – up to 8192 entries 12 November 2013 How does the processor find the GDT/LDT? University of Virginia cs4414 27
  • 29.
    The GDT andLDT are just data structures in memory! Special registers store their locations 12 November 2013 University of Virginia cs4414 28
  • 30.
    Logical Address Segmentatio n Unit LinearAddress Paging Unit Physical Address Memory 12 November 2013 University of Virginia cs4414 29
  • 31.
    Linear Address Logical Address 64 2 Segmentation Unit Paging linearaddresses What would it cost to have 264 bytes of RAM? 12 November 2013 University of Virginia cs4414 30
  • 32.
    $10 per 1GB= 230 bytes 264 bytes = $10 * 234 $172B ~ ½ Google’s Market Cap 12 November 2013 University of Virginia cs4414 31
  • 33.
    Paging Memory Paging Unit Physical Address Linear Address LogicalAddress Segmentation Unit We don’t need to store the whole address space in memory! Most of it is unused, and we can store rarely-used parts on the disk. 12 November 2013 University of Virginia cs4414 32
  • 34.
    Memory Physical Address Linear Address Paging Unit Imagefrom Wikipedia 12 November 2013 University of Virginia cs4414 33
  • 35.
    Overview (Intel 386) 32-bitlinear address CR3 Dir Page 10 bits (1K tables) Page Directory Offset 10 bits 12 bits (1K entries) (4K pages) Page Entry Page Table Physical Memory Page + Offset CR3+Dir 12 November 2013 University of Virginia cs4414 34
  • 36.
    Page Table Entries CR3 PageEntry Page Table Page Directory 12 November 2013 20 bits: physical address of page 12 bits: flags user/kernel page write permission present University of Virginia cs4414 Physical Memory Page + Offset 35
  • 37.
    386 Checkup 32-bit linearaddress CR3 Dir Page 10 bits (1K tables) Page Directory Offset 10 bits 12 bits (1K entries) (4K pages) 20 bits addr / 12 bits flags Page Table Physical Memory Page + Offset CR3+Dir How many pages do we need to store the page table? 12 November 2013 University of Virginia cs4414 36
  • 38.
    How slow isthis???! Memory Page Paging Unit Physical Address Dir Linear Address Logical Address Segmentation Unit Offset CR3 Page Directory Page Table Physical Memory GDTR Global Descriptor Table 12 November 2013 University of Virginia cs4414 37
  • 39.
    Translation Lookaside Buffer(Cache) Memory Page Paging Unit Physical Address Dir Linear Address Logical Address Segmentation Unit Offset CR3 Page Directory Page Table Physical Memory GDTR Global Descriptor Table 12 November 2013 University of Virginia cs4414 38
  • 40.
    Page Fault CR3 Page Entry PageTable Page Directory 12 November 2013 Physical Memory 20 bits: physical address of page 12 bits: flags user/kernel page write permission present University of Virginia cs4414 39
  • 41.
    How common arepage faults? 12 November 2013 University of Virginia cs4414 40
  • 42.
    top -o mem-stats pid,command,cpu,mem,mregion,vsize,faults 12 November 2013 University of Virginia cs4414 41
  • 43.
    Physical Memory: 9106M used(2553M “wired” – cannot be paged out) + 5090M unused = 14196M where is my missing ~2GB???! Virtual Memory: 594G (total) top -o mem -stats pid,command,cpu,mem,mregion,vsize,faults 12 November 2013 University of Virginia cs4414 42
  • 44.
    How expensive isa page fault? 12 November 2013 University of Virginia cs4414 43
  • 45.
    #include <stdio.h> #include <stdlib.h> intmain(int argc, char **argv) { char *s = (char *) malloc (1); int i= 0; while (1) { printf("%d: %xn", i, s[i]); i += 4; } } 12 November 2013 What will this program do? > ./a.out 0: 0 4: 0 8: 0 12: 0 …1033872: 0 1033876: 0 1033880: 0 1033884: 0 Segmentation fault: 11 University of Virginia cs4414 44
  • 46.
    Charge • Make progresson your projects: everyone should have a clear idea what you are doing now • Will post more details on next deliverable (design reviews) soon Challenge: write a program that takes N as an input and produces (nearly) exactly N page faults. 12 November 2013 University of Virginia cs4414 45