MIT 6.1810 2023

Schedule site https://pdos.csail.mit.edu/6.828/2023/schedule.html

Study blog for mit 6.1810 https://www.jianshu.com/nb/54437584

folder lab1-10 is the solution file for each lab

xv6-all is a runnable xv6 os that has completed all the optional challenges in the course.

Target

1. Base Correctness

I combined all the existing tests into one file ./grade-lab-all to ensure the os can pass all existing tests

2. Optional Challenge Correctness

I wrote a lot of test cases for each optional challenge task ./grade-lab-oc to ensure all oc can be passed

List

Lab 1 (util)

Write an uptime program that prints the uptime in terms of ticks using the uptime system call. (easy)
Support regular expressions in name matching for find. grep.c has some primitive support for regular expressions. (easy)
The xv6 shell (user/sh.c) is just another user program and you can improve it. It is a minimal shell and lacks many features found in real shell. For example, modify the shell to not print a $ when processing shell commands from a file (moderate), modify the shell to support wait (easy), modify the shell to support lists of commands, separated by ";" (moderate), modify the shell to support sub-shells by implementing "(" and ")" (moderate), modify the shell to support tab completion (easy), modify the shell to keep a history of passed shell commands (moderate), or anything else you would like your shell to do. (If you are very ambitious, you may have to modify the kernel to support the kernel features you need; xv6 doesn't support much.)

Lab 2 (syscall)

Print the system call arguments for traced system calls (easy).
Compute the load average and export it through sysinfo(moderate).

Lab 3 (pgtbl)

Use super-pages to reduce the number of PTEs in page tables. (to avoid break other tests, only mmap is allowed to use super page)
Unmap the first page of a user process so that dereferencing a null pointer will result in a fault. You will have to change user.ld to start the user text segment at, for example, 4096, instead of 0. Add a system call that reports dirty pages (modified pages) using PTE_D.

Lab 4 (traps)

Print the names of the functions and line numbers in backtrace() instead of numerical addresses (hard).

Lab 5 (cow)

Modify xv6 to support both lazy page allocation and COW.
Measure how much your COW implementation reduces the number of bytes xv6 copies and the number of physical pages it allocates. Find and exploit opportunities to further reduce those numbers.

Lab 6 (thread)

The user-level thread package interacts badly with the operating system in several ways. For example, if one user-level thread blocks in a system call, another user-level thread won't run, because the user-level threads scheduler doesn't know that one of its threads has been descheduled by the xv6 scheduler. As another example, two user-level threads will not run concurrently on different cores, because the xv6 scheduler isn't aware that there are multiple threads that could run in parallel. Note that if two user-level threads were to run truly in parallel, this implementation won't work because of several races (e.g., two threads on different processors could call thread_schedule concurrently, select the same runnable thread, and both run it on different processors.) There are several ways of addressing these problems. One is using scheduler activations and another is to use one kernel thread per user-level thread (as Linux kernels do). Implement one of these ways in xv6. This is not easy to get right; for example, you will need to implement TLB shootdown when updating a page table for a multithreaded user process. Add locks, condition variables, barriers, etc. to your thread package.

Lab 7 (network)

In this lab, the networking stack uses interrupts to handle ingress packet processing, but not egress packet processing. A more sophisticated strategy would be to queue egress packets in software and only provide a limited number to the NIC at any one time. You can then rely on TX interrupts to refill the transmit ring. Using this technique, it becomes possible to prioritize different types of egress traffic. (easy)
The provided networking code only partially supports ARP. Implement a full ARP cache and wire it in to net_tx_eth(). (moderate)
The E1000 supports multiple RX and TX rings. Configure the E1000 to provide a ring pair for each core and modify your networking stack to support multiple rings. Doing so has the potential to increase the throughput that your networking stack can support as well as reduce lock contention. (moderate), but difficult to test/measure. (I don't find clues in e1000 doc to support this; Let me know if you have)
sockrecvudp() uses a singly-linked list to find the destination socket, which is inefficient. Try using a hash table and RCU instead to increase performance. (easy), but a serious implementation would difficult to test/measure
ICMP can provide notifications of failed networking flows. Detect these notifications and propagate them as errors through the socket system call interface. (I support ICMP ping)
The E1000 supports several stateless hardware offloads, including checksum calculation, RSC, and GRO. Use one or more of these offloads to increase the throughput of your networking stack. (moderate), but hard to test/measure
The networking stack in this lab is susceptible to receive livelock. Using the material in lecture and the reading assignment, devise and implement a solution to fix it. (moderate), but hard to test.
Implement a UDP server for xv6. (moderate)
Implement a minimal TCP stack and download a web page. (hard)

Lab 8 (lock)

maintain the LRU list so that you evict the least-recently used buffer instead of any buffer that is not in use.
make lookup in the buffer cache lock-free. Hint: use gcc's _sync* functions. How do you convince yourself that your implementation is correct? (I use a lock-free queue and a lock-free array-based map and CAS to achieve it)

Lab 9 (fs)

Support triple-indirect blocks. (only need change some config to support)

Lab 10 (mmap)

If two processes have the same file mmap-ed (as in fork_test), share their physical pages. You will need reference counts on physical pages.
Your solution probably allocates a new physical page for each page read from the mmap-ed file, even though the data is also in kernel memory in the buffer cache. Modify your implementation to use that physical memory, instead of allocating a new page. This requires that file blocks be the same size as pages (set BSIZE to 4096). You will need to pin mmap-ed blocks into the buffer cache. You will need worry about reference counts.
Remove redundancy between your implementation for lazy allocation and your implementation of mmap-ed files. (Hint: create a VMA for the lazy allocation area.)
Modify exec to use a VMA for different sections of the binary so that you get on-demand-paged executables. This will make starting programs faster, because exec will not have to read any data from the file system.
Implement page-out and page-in: have the kernel move some parts of processes to disk when physical memory is low. Then, page in the paged-out memory when the process references it. (Allocate specific region in fs for paging; The memory page eviction is the aging algorithm）

yixuaz / 6.1810-2023