Code generation and memory allocation
magmanu opened this issue · comments
Yes, I promise I've read the Contributions Guidelines (please feel free to remove this line).
Please type "I already searched for this issue": I already searched for this issue
Edition: (1st or 2nd) 2nd
Book Title: Scope & Closures
Chapter: 1 & 2
Section Title: Compiling Code
Question:
Thanks for this excellent book, I'm enjoying every line of it!
Could you please address a question?
In Chapter 1, section Compiling Code states: "The JS engine takes the just described AST for var a = 2; and turns it into a set of machine instructions to actually create a variable called a (including reserving memory, etc.), and then store a value into a."
However, in chapter 2, section A Conversation Among Friends, memory allocation seems to be described as happening at execution time:
"Encountering var students, Compiler will ask Scope Manager to see if a variable named students already exists for that particular scope bucket. If so, Compiler would ignore this declaration and move on. Otherwise, Compiler will produce code that (at execution time) asks Scope Manager to create a new variable called students in that scope bucket."
Could you please clarify this apparent contradiction?
Is it the case that when the compiler reserves memory, it reserves it as an empty, anonymous slot that will be later filled by the scope manager with the actual variable name and variable value? If yes, how does this mapping between reserved memory and actual allocation happen? Could it happen that the reserved memory is not large enough for the actual value that has to be stored? How would JS solve this problem?
Many thanks,
Manu
The compiler never does "memory allocation" in the strict sense, since the compiler is not RUNNING your code. In a C++ program, the C++ compiler obviously cannot reserve memory for an executable that will run much later, and potentially run on different machines!
Instead, the compiler produces instructions inside the executable that will reserve memory when the executable is later run.
In chapter one, "turns it into a set of machine instructions" means, create the instructions, not run the instructions at that moment. In chapter two, "at execution time" reinforces that the instructions ("code") the compiler produces earlier is not actually executed, to reserve the memory, until execution time (aka "run time").
Hopefully that's clear now.
As for "what happens if the program needs more memory than compiler's instructions told it to reserve?", the answer is Stack vs Heap. Compiler reserves (sets up instructions to do so, at run-time) memory for known needs, like the number 42, on the Stack. It also puts in instructions for dynamic memory allocation during run-time for things it cannot know in advance, such as extending an array, which allocates memory off the Heap.
If you're curious about those details, there's lots to google about Stack vs Heap memory allocations in computer science. Here's a link I just found: https://www.geeksforgeeks.org/stack-vs-heap-memory-allocation/ It talks about Stack vs Heap in the context of C++, not JS... but the principles are pretty similar.
Thank you so much, Kyle, it does clarify :)
And thanks for pointing me to Stack vs Heap, it's the first time I hear about that. Found some articles that also explained memory leak in JS (I was also curious about that) and things are making more sense. Thanks once again!
Hi! Just wanted to ask some questions if I'm understanding this correctly (I wasn't sure if I should have made a new issue, but it seemed more related to this issue). In chapter 2, under nested scope section, you mention this:
Each scope automatically has all its identifiers registered at the start of the scope being executed
By registered, do you mean identifiers are created at the start of the scope execution?
In chapter 5, under when can i use variable, you mention this:
Recall Chapter 1 points out that all identifiers are registered to their respective scopes during compile time. Moreover, every identifier is created at the beginning of the scope it belongs to, every time that scope is entered.
So just to group together the terms used in the book by their meaning (how I understood it) (Also, I feel bad for skipping a lot of contexts of the book and quoting only sections of it):
Create machine instruction that will later create variables in code execution:
- Compiler registering identifier in compile/parsing phase (chapter 2, nested scope & chapter 5, when can i use variable)
- Compiler creating map of lexical scopes (chapter 1, under lexical scope)
- Scope manager creating identifier (chapter 2, under A Conversation Among Friends)
- Compiler sets up the declaration of the scope variable (chapter 2, under A Conversation Among Friends)
Create actual identifier in memory:
- Scope automatically having all its identifiers registered at the start of the scope (chapter 2, under nested scope)
So Javascript engine will:
- Parse through the code
- Create machine instruction whenever it encounter identifier declaration (var/let/const)
- Start executing the code
- At the very beginning of scope execution, create that previously encountered identifiers (of the current scope) in memory
- Assign value to that created identifiers depending on how they are declared (var -> undefined, function -> function reference, let, const -> uninitialized)
Is this correct?
Seems like a reasonable summary. :)
The apparent contradiction between the two statements can be resolved by understanding the different stages of the JavaScript code execution process.
In Chapter 1, section Compiling Code, it describes the first stage, which is the compilation phase. During this phase, the JavaScript engine takes the source code, performs lexical analysis and creates an Abstract Syntax Tree (AST). The engine then uses this AST to generate machine code or bytecode, which includes reserving memory for variables, function declarations, etc. In this stage, the engine knows the variable name (e.g., "a") and the value (e.g., 2) that will be assigned to it. Therefore, it can reserve the appropriate memory for that variable and assign the value.
However, in Chapter 2, section A Conversation Among Friends, it describes a more detailed process of variable declaration at execution time, during the runtime or execution phase. In JavaScript, there is a separation between the compilation phase and the execution phase.
When the engine encounters a variable declaration like var students; during the execution phase, it delegates the actual creation of the variable to the Scope Manager. The Scope Manager handles the scope in which the variable is declared and manages the variables' lifecycle. At this point, the engine doesn't know the actual value that will be assigned to the variable (if any) but only that a variable named "students" has been declared.
Regarding your question about memory allocation, when the engine reserves memory during the compilation phase, it usually allocates a fixed-size slot for each variable. This size is determined by the data type of the variable or a default size for variables of unknown type. When the value is later assigned to the variable at execution time, the engine will take care of handling any size requirements or conversions. If the value to be stored is too large for the reserved memory slot, the engine will dynamically allocate more memory to accommodate the value. This process is usually transparent to the developer, as JavaScript handles memory management automatically.
So, to summarize:
1.During the compilation phase, the JavaScript engine reserves memory for variables based on the known types and values in the source code.
2.During the execution phase, the actual variable creation is performed by the Scope Manager, and the engine assigns values to those variables as needed.
3.If the reserved memory is not large enough to hold the actual value, the engine will allocate more memory to accommodate it.
JavaScript's memory management and allocation are abstracted from developers, making it easier to work with variables without worrying about low-level memory details.