ablaamim/Webserver

Webserv :

🔥 grade 🔥

⚡ Collaborators ⚡

This project is about writing a HTTP server, your own version of Nginx, Apache or Lighttpd.

What is Webserv ?

Webserv is a group project at 1337 which requires us to build a HTTP server using I/O Multiplexing built on top of a event loop.

In the mandatory part, we start the server by reading a configuration file that can be based on Nginx, containing information like hostname(ip/port), servername, index file, directory listing, URL redirect, location(vhost path) and some other settings. It is forbidden to use fork to create new non-CGI processes or spawn threads (threads are not available in C++98).

Ressources

Playlist

General instructions

Your program must not crash under any circumstances (even if it runs out of memory) and must not terminate unexpectedly. If this happens, your project will be considered non-functional and your score will be 0.
You must include a Makefile that will compile your source files. It should not re-link.
Your Makefile must at least contain the rules:

$(NAME), all, clean, fclean and re
Compile your code with c++ and the flags -Wall -Wextra -Werror
Your code must comply with the C++ 98 standard. Then, it should still compile if you add the flag -std=c++98.
Try to always develop using as many C++ features as possible (for example, choose over <string.h>). You are allowed to use C functions, but always prefer their C++ versions if possible.
Any external library and Boost libraries are prohibited

Mandatory part

Program name	web server
Files for donation	Makefile, .{h, hpp}, .cpp, .tpp, .ipp, configuration files
Makefile	NAME, all, clean, fclean, re
Options	[A configuration file]
Allowed external functions	Everything in C++ 98 , getsockname, fcntl
Using libft	n/a
Description	A HTTP server in C++ 98

You must write a HTTP server in C++ 98.

Your executable will be run as follows: ./webserv [configuration file]

Even if the subject and rating scale mention poll(), you can use any equivalent such as select(), kqueue(), or epoll().

Please read the RFC and run some tests with telnet and NGINX before starting this project. Even if you don't need to implement every RFC, reading it will help you develop the features you need.

Subject && Requirements

Your program must accept a configuration file as an argument, or use the default path.
You cannot execve another web server
Your server should never block and the client can be properly returned if needed.
It must be non-blocking and use only 1 poll() (or equivalent) for all I/O operations between client and server (including listening).
poll() (or similar) should check for read and write at the same time.
You should never perform a read or write operation without going through poll() (or similar).
Checking the value of errno is strictly prohibited after a read or write operation.
You don't need to use poll() (or equivalent) before reading the config file.

Because you have to use non-blocking file descriptors, you can use the read/get or write/send functions without poll() (or equivalent) and your server won't block. But it will consume more system resources. Thus, if you try to read/get or write/send any file descriptor without using poll() (or similar), your score will be 0.

You can use any macro and define as FD_SET, FD_CLR, FD_ISSET, FD_ZERO (understanding what and how they do is very helpful).
A request to your server should never hang forever.
Your server must be compatible with the web browser of your choice.
We will assume that NGINX is compatible with HTTP 1.1 and can be used to compare headers and behavior of responses.
Your HTTP response status codes must be accurate.
Your server should have default error pages if not provided.
You cannot use fork for anything other than CGI (like PHP or Python, etc.).
You must be able to maintain a completely static website.
Clients must be able to upload files.
You need at least the GET, POST and DELETE methods.
Stress tests of your server. It must be available at any cost.
Your server must be able to listen on multiple ports (see configuration file).

For MacOS only

Since MacOS does not implement write() in the same way as other Unix OSes, you are allowed to use fcntl(). You must use file descriptors in non-blocking mode to get behavior similar to other Unix operating systems.

However, you are only allowed to use fcntl() like this: fcntl(fd, F_SETFL, O_NONBLOCK); Any other flags are prohibited.

Configuration file

You can get some inspiration from the "server" part of the NGINX configuration file.

In the config file you should be able to:

Select the port and host of each server.
Set server_names or not.
The first server for host:port will be the default for that host:port (meaning it will respond to all requests that do not belong to another server).
Customize default error pages.
Limit body size client.
Set up routes with one or more of the following rules/configurations (routes will not use regex):
- Define a list of valid HTTP methods for the route.
- Define HTTP redirect
- Specify the directory or file in which to look for the file (for example, if the URL /kapouet is linked to /tmp/www, the URL /kapouet/pouic/toto/pouet is /tmp/ www/pouic/toto/pouet).
- Turn the directory listing on or off.
- Execute CGI based on a specific file extension (eg .php).
- Make it possible for the route to accept downloaded files, and configure where they are saved.
  - Are you wondering what is CGI?
  - Since you won't be calling CGI directly, use the full path as PATH_INFO.
  - Just remember that for a fragmented request, your server must unblock it, and CGI will expect EOF as the end of body.
  - Same for CGI output. If no content_length is returned from CGI, EOF marks the end of the returned data.
  - Your program must call CGI with the requested file as the first argument.
  - CGI must run in the correct directory to access relative path files.
  - Your server must work with one CGI (php-CGI, Python, etc.).

You must provide some configuration files and default base files for testing and demonstrating how each feature works during evaluation.

If you have a question about one behavior, you should compare your program's behavior with that of NGINX. For example, check how server_name works. We have shared with you a small tester. It is not necessary to go through it if everything works fine with your browser and tests, but it can help you find some errors.

The main thing is sustainability. Your server should never die.

Don't test with just one program. Write tests in a more convenient language like Python or Golang etc. Even in C or C++ if you want.

Bonus part

Here are additional features you can add:

Support for cookies and session_management files (prepare short examples).
Handling multiple CGI.

The bonus part will only be evaluated if the required part is "EXCELLENT". Perfect means that the required part has been fully completed and is running smoothly. If you have not met ALL of the mandatory requirements, your bonus portion will not be evaluated at all.

Dependencies :

brew install php --with-cgi --with-debug

🔁 Parsing Logic : DOCUMENTATION !

Author : ablaamim

Parsing configuration file :

Specifications restricted by the subject :

In the configuration file, you should be able to:

• Choose the port and host of each ’server’.

• Setup the server_names or not.

• The first server for a host:port will be the default for this host:port (that means

it will answer to all the requests that don’t belong to an other server).

• Setup default error pages.

• Limit client body size.

• Setup routes with one or multiple of the following rules/configuration (routes wont

be using regexp):

◦ Define a list of accepted HTTP methods for the route.

◦ Define a HTTP redirection.

◦ Define a directory or a file from where the file should be searched (for example,

if url /kapouet is rooted to /tmp/www, url /kapouet/pouic/toto/pouet is /tmp/www/pouic/toto/pouet).

◦ Turn on or off directory listing.

◦ Set a default file to answer if the request is a directory.

◦ Execute CGI based on certain file extension (for example .php).

◦ Make it work with POST and GET methods.

◦ Make the route able to accept uploaded files and configure where they should be saved.

Configuration file specifications :

Configuration file is a text file that contains various settings and directives that dictate how the web server should operate. These settings can include things like the port number that the web server should listen on, the location of the web server's root directory, and many other settings.

Here is an example fie that shows config file format and supported directives.

server 
{
    listen 127.0.0.1 8001;              # listening ip/port
    error_pages 404 /error/404.html;    # default error page
    max_body_size 1024;                 # max request body size in bytes
    index index.html index2.html;       # default page when requesting a directory, index.html by default

    location /
    {                   
        root Webserver/www/html;        # root folder of site directory, full or relative path
        auto_index on;                  # turn on/off directory listing
        allowed_methods POST GET;       # allowed methods in location, only get is set by default
        index index.html;               # default page when requesting a directory, copies root index by default
        return abc/index1.html;         # redirection
    }

    location /cgi-bin
    {
        root root Webserver/www/        # root folder of cgi
        bin-cgi .py /usr/bin/python3;   # location of interpreters installed on the current system, mandatory parameter and extensions for executable files, mandatory parameter
    }
}

🔧 Testing 🔧 :

TEST TO RUN	PART	ERROR TYPE	CONFIG TO TEST WITH / Command	AUTHOR	STATUS
Makefile	Program	No relink	Makefile in root	Team	✅
Take a configuration file as argument or use default path	Program	Should run with or without configuration file	None	Abdessamad	✅
Siege & stress test	Multiplexing	Siege Result is 100% Also server never hangs (Always Available)	siege -b localhost:8080	Zineb	✅
You should be able to use siege indefinitely without having to restart the server (take a look at siege -b)	Multiplexing	Working	siege -b localhost:8080	Zineb	✅
Multiple Listen inside same server-context	Parsing	each server could bind more than one socket	config_file	Abdessamad	✅
No index and no auto_index provided in conf	Response	Segmentation fault	config_file	Achraf	✅
No root in location context	Parsing	Unapropriate behavior	config_file	Abdessamaad	✅
Protection of all system calls	The whole program	Exceptions must be thrown	None needed	Team	✅
HTTP Response Code 413 (Content too large)	Request check	Error code not accurate	POSTMAN + config_file	Zineb	✅
HTTP Response Code 411 (Content length required)	Request check	Error code not accurate	POSTMAN + config_file	Zineb	✅
HTTP Response Code 400 (Bad Request)	Request check	Error code not accurate	POSTMAN + config_file	Zineb	✅
HTTP Response Code 501 (Not implemented) for Transfert encoding not chuncked	Request check	Error code not accurate	config_file	Zineb	✅
Setup multiple Servers with different ports	Socket creation and Parsing	Working	config_file	Abdessamad	✅
Setup multiple servers with different hostnames/server_names	Program	Working	config_file + curl http://www.example.com --resolve www.example.com:8080:127.0.0.1	team	✅
Limit the client body	-	Working	(use: curl -X POST -H "Content-Type: plain/text" --data "BODY IS HERE write something shorter or longer than body limit")	-	✅
Setup routes in a server to different directories	Parsing/Response	Working	config_file	Abdessamad/Achraf	✅
Setup a list of methods accepted for a certain route [GET]	Response	Get method on a video doesnt render it	config_file	Achraf	✅
Try to list a directory	Response	Working	config_file	Abdessamad	✅
Setup multiple Servers with same configuration, the first server must be the default to select	Parsing/Response	Working	config_file	Achraf/Abdessamad	✅
Verify there is no memory leak (Monitor the process memory usage. It should not go up indefinitely)	Program	-	leaks.sh + config_file	Team	✅
Set one server_name per server, stop the program otherwise	Parsing	Need to throw an exception	config_file	Abdessamad	✅
UNKNOWN requests should not result in a crash	Check request	Accurate error code	POSTMAN + config_file	Abdessamad	✅

ablaamim / Webserver