Youth Training Camp | "Node.js and Front-end Development Practice"

Key Content of This Lesson#

Application Scenarios of Node.js (why)#

Front-end Engineering
- Early libraries like jQuery were directly included in the page, but as modularization matured, Node.js gave developers the ability to run code outside the browser, leading to gradual modularization in the front-end.
- Bundle: webpack, Vite, esbuild, Parcel etc.
- Uglify: UglifyJS
- Transpile: babeljs, TypeScript
  
  My personal understanding: Transpiling means converting the latest syntax like ES6 into lower version syntax to achieve browser compatibility.
- Other languages joining the competition in front-end engineering: esbuild, Parcel, prisma, etc.
- Current situation: Node.js is hard to replace.
Web Server Applications
- Gentle learning curve, high development efficiency
- Running efficiency close to common compiled languages
- Rich community ecosystem and mature toolchain (npm, V8 inspector)
- Advantages in scenarios combined with front-end (SSR, isomorphic front-end applications. Both page writing and backend data fetching and filling are done by JavaScript)
- Current situation: fierce competition, Node.js has its unique advantages.
Electron cross-platform desktop applications
- Commercial applications: vscode, slack, discord, zoom
- Efficiency tools within large companies
- Current situation: most scenarios are worth considering during selection.

Node.js Runtime Structure (what)#

N-API: Some packages installed using npm in user code.
V8: JavaScript Runtime, diagnostic debugging tool (inspector)
libuv: event loop, syscall
- Example: When making a request using node-fetch
- The underlying process will call a lot of C++ code.

Characteristics

Asynchronous I/O:
```
setTimeout(() => {
    console.log('B');
})
console.log('A');
```
- A common scenario: Reading a file. When Node.js performs I/O operations, it will resume operations after the response returns, rather than blocking the thread and occupying extra memory while waiting. (Less memory usage)
Single-threaded
- worker_thread can start an independent thread, but the model for each thread does not change much.
```
function fibonacci(num:number):number {
	if(num === 1 || num === 2) {
        return 1;
    }
    return fibonacci(num-1) + fibonacci(num-2);
}
fibonacci(42)
fibonacci(43)
```
- JS single-threaded
  - Actual: JS thread + uv thread pool (4 threads) + V8 task thread pool + V8 Inspector thread
- Advantages: No need to consider multi-threaded state synchronization issues, thus no locks are needed. At the same time, it can utilize system resources efficiently;
- Disadvantages: Blocking can have more negative impacts, asynchronous issues, scenarios with timing requirements need to be considered.
  - Solution: Multi-process or multi-threading.
Cross-platform (most functionalities, APIs)
- If you want to use a Socket on Linux, and the calls are different across platforms, you only need:
```
const net = require('net')
const socket = new net.Socket('/tmp/socket.sock')
```
- Node.js is cross-platform + JS does not require a compilation environment (+ Web is cross-platform + diagnostic tools are cross-platform)
  - = Low development cost (most scenarios do not need to worry about cross-platform issues), overall low learning cost.

Writing an Http Server (how)#

Installing Node.js#

Mac, Linux recommend using nvm. Multi-version management.
Windows recommend nvm4w or the official installation package.
If installation is slow or fails, set the installation source:
- NVM_NODEJS_ORG_MIRROR=https://npmmirror.com/mirrors/node nvm install 16

Writing Http Server + Client, Sending and Receiving GET, POST Requests#

Prerequisite: Node.js is installed, open cmd with administrator privileges and navigate to the current file directory.

Http Server#

First, write a server.js as follows:

createServer explanation

req request, res response

server.listen explanation

port is the port number to listen to, callback function after success.

const http = require('http');
const server = http.createServer((req, res) => {
    res.end('hello'); // The response is directly hello
});
const port = 3000;
server.listen(port, () => {
    console.log(`server listens on:${port}`);	// Listening on port 3000
})

Start with node, at this point entering localhost:3000 will show hello.

Change to JSON version:

  const server = http.createServer((req, res) => {
      // receive body from client
      const bufs = [];    // Get the transmitted data
      req.on('data', data => {
          bufs.push(data);
      });
      req.on('end', () => {
          const buf = Buffer.concat(bufs).toString('utf-8');
          let msg = 'Hello';
          try {
              reqData = JSON.parse(buf);
              msg = reqData.msg;
          } catch (err) {
              res.end('invalid json');
          }
          // response
          const responseJson = {
              msg: `receive：${msg}`
          }
          res.setHeader('Content-Type', 'application/json');
          res.end(JSON.stringify(responseJson));
      });
  });

Http Client#

http.request(url[, options\][, callback])

const http = require('http');
const body = JSON.stringify({ msg: 'hello from my own client' });
// [url] [option] [callback]
const req = http.request('http://127.0.0.1:3000', {
    method: 'POST',
    headers: {
        'Content-Type': 'application/json',
        'Content-Length': body.length,
    },
}, (res) => {   // Response body   
    const bufs = [];
    res.on('data', data => {
        bufs.push(data);
    });
    res.on('end', () => {
        const buf = Buffer.concat(bufs);
        const receive = JSON.parse(buf);
        console.log('receive json.msg is:', receive);
    });
})
req.end(body);

Promisify#

You can rewrite these two examples using Promise + async and await (why?)

The true advantage of the await keyword becomes apparent when used with asynchronous functions — in fact, await only works inside asynchronous functions. It can be placed before any asynchronous, promise-based function. It will pause the code at that line until the promise completes, then return the result value. While paused, other code waiting to execute has the opportunity to run.

async/await makes your code look synchronous, and to some extent, makes its behavior more synchronous. The await keyword blocks the subsequent code until the promise completes, just like executing synchronous operations. It does allow other tasks to continue running in the meantime, but your own code is blocked.

Too many callbacks make it hard to find and maintain.

function wait(t) {
    return new Promise((resolve, reject) => {
        setTimeout(() => {
            resolve();
        }, t);
    });
}
wait(1000).then(() => { console.log('get called'); });

Not all callbacks are suitable for rewriting as Promises.

Suitable for callbacks that are only called once.

const server = http.createServer(async (req, res) => {  // Note the async here
    // receive body from client changed to Promise form
    const msg = await new Promise((resolve, reject) => {    // Execute and then assign to msg
        const bufs = [];    
        req.on('data', data => {
            bufs.push(data);
        });
        req.on('error', (err) => {
            reject(err);
        })
        req.on('end', () => {
            const buf = Buffer.concat(bufs).toString('utf-8');
            let msg = 'Hello';
            try {
                reqData = JSON.parse(buf);
                msg = reqData.msg;
            } catch (err) {
                // 
            }
            resolve(msg);
        });
    });
    // response
    const responseJson = {
        msg: `receive：${msg}`
    }
    res.setHeader('Content-Type', 'application/json');
    res.end(JSON.stringify(responseJson));
});

Writing a Static File Server#

Write a simple static service that accepts HTTP requests from users, retrieves the URL of the image, which is agreed to correspond to the path on the static file server's disk, and then returns the specific content to the user. This time, in addition to the http module, you will also need the fs module and the path module.

First, write a simple index.html and place it in the static directory.

static_server.js#

const http = require('http');
const fs = require('fs');
const path = require('path');
const url = require('url');
// __dirname is the location of the current file, ./ is the folder where the current file is located, folderPath is the relative path to the static folder from the current file.
const folderPath = path.resolve(__dirname, './static');

const server = http.createServer((req, res) => {  // Note the async here
    // expected http://127.0.0.1:3000/index.html
    const info = url.parse(req.url);

    // static/index.html
    const filepath = path.resolve(folderPath, './'+info.path);
    console.log('filepath', filepath);
    // Stream-style API, better memory usage internally.
    const filestream = fs.createReadStream(filepath);
    filestream.pipe(res);
});
const port = 3000;
server.listen(port, () => {
    console.log(`server listens on:${port}`);
})

What else is needed compared to high-performance, reliable services?

CDN: Caching + Acceleration
Distributed storage, disaster recovery (the server can still serve normally even if it goes down).

Writing a React SSR Service#

What are the characteristics of SSR (server-side rendering)?
Compared to traditional HTML template engines: avoids repetitive code writing.
Compared to SPA (single-page application): faster initial rendering, SEO (search engine optimization) friendly.
Disadvantages:
- Typically lower qps (queries per second), front-end code writing needs to consider server-side rendering situations.
- More complex to write, JavaScript writing also needs to consider presentation in the front end.

Installing React#

npm init
npm i react react-dom

Writing an Example#

const React = require('react');
const ReactDOMServer = require('react-dom/server');
const http = require('http');
function App(props) {
    return React.createElement('div', {}, props.children || 'Hello');
}
const server = http.createServer((req, res) => {
    res.end(`
        <!DOCTYPE html>
        <html>
            <head>
                <title>My Application</title>
            </head>
            <body>
                ${ReactDOMServer.renderToString(
                    React.createElement(App, {}, 'my_content'))}
                <script>
                    alert('yes');
                </script>
            </body>
        </html>
    `);
})
const port = 3000;
server.listen(port, () => {
    console.log('listening on: ', port);
})

Challenges of SSR

Need to handle code packaging.
Need to think about the logic of front-end code running on the server.
Remove side effects that are meaningless on the server, or reset the environment.

Using Inspector for Debugging and Diagnosis#

V8 Inspector: out-of-the-box, feature-rich and powerful, consistent with front-end development, cross-platform.
- node -- inspect
- open http://localhost:9229/json
Scenarios:
- View console.log content.
- Breakpoints.
- High CPU, infinite loops: cpuprofile.
- High memory usage: heapsnapshot (heap snapshot).
- Performance analysis.

Overview of Deployment#

After writing, how to deploy to the production environment?

Problems to solve in deployment:
- Daemon: Restart when the process exits.
- Multi-process: cluster conveniently utilizes multiple processes.
- Record process status for diagnosis.
Container environment:
- Usually has health check methods, only need to consider multi-core CPU utilization.

Extended Topics#

Quickly Understanding Node.js Code#

Node.js Core Contribution Guide

Benefits:
- Gradually understand the underlying details from the user's perspective, which can solve more complex problems.
- Self-validation, helpful for career development.
- Solve community issues and promote community development.
Challenges:
- Time-consuming (realistic).

Compiling Node.js#

Why learn to compile Node.js?
- Cognition: from black box to white box, able to trace problems when they occur.
- The first step in contributing code.
How to compile:
- ./configure && make install
- Demonstration: adding custom properties to the net module.

Diagnosis/Tracing#

Diagnosis is a low-frequency, important, and quite challenging direction. It is an important reference for enterprises to measure whether they can rely on a programming language.
A popular role in the technical consulting industry.
Challenges:
- Need to understand the underlying Node.js, need to understand the operating system and various tools.
- Requires experience.

WASM, NAPI#

Node.js (because of V8) is a natural container for executing WASM code, and the runtime is the same as the browser's WASM, while Node.js supports WASI.
NAPI executes C interface code (C/C++/Rust...), while retaining the performance of native code.
A solution for communication between different programming languages.

Summary and Thoughts#

This lesson starts with an introduction to Node.js, implementing a practical example of writing an Http Server (with Promise optimization for callbacks, and gaining some understanding of SSR), and the teacher also provides some suggestions and further reading in the extended topics, which is great~

Most of the content referenced in this article comes from Teacher Ouyang Yadong's class and MDN.