Networking

Access to core networking functionality can be obtained by using this.context.connection from a component. The default backend server connects users to rooms. Users in the same room will share state and receive messages from each other.

Networking is currently based on websockets and sending either json strings (for infrequent updates) or flatbuffers (for frequent updates). Continue reading below for more details:

Using Multiplayer

• Enable Networking
  Add a SyncedRoom component.

• Enable Desktop Viewer Sync
  Add a SyncedCamera component.

• Enable XR Avatar Sync
  Add a WebXRSync component.

• Enable Voice Chat
  Add a VoIP component.

• Enable Screensharing
  Add a ScreenCapture component.

Core Components

• SyncedRoom — handles networking connection and connection to a room.
  This can also be done by code using the networking api accessible from this.context.connection
• SyncedTransform — handles synchronizing transforms
• SyncedCamera — spawns a prefab for any user connected to the room which will follow their position
• WebXRSync — handles synchronization for AR and VR users
• VoIP — handles voice-over-IP audio connections, microphone access etc. between users
• Networking — use to customize the server backend url

Manual Networking

Sending

Send a json message to all users in the same room:
this.context.connection.send(key:string, data: IModel | object | boolean | string | number | null)

Send a flatbuffer binary array to all users in the same room:
this.context.connect.sendBinary(arr:Uint8Array)

Persistence

When sending an object containing a guid field it will be saved in the persistant storage and automatically sent to users that connect later or come back later to the site (e.g. to restore state).
To delete state for a specific guid from the backend storage you can use delete-state as the key and provide an object with { guid: "guid_to_delete" }

Receiving

Subscribe to json events / listen to events in the room using a specific key
this.context.connection.beginListen(key:string, callback:(data) => void)
Unsubscribe with stopListening

Subscribe to flatbuffer binary events
this.context.connection.beginListenBinrary(identifier:string, callback:(data : ByteBuffer) => void)
Unsubscribe with stopListenBinary

Auto Networking (experimental)

To automatically network fields in a component you can just decorate a field with a @syncField() decorator (note: you need to have experimentalDecorators: true in your tsconfig.json file for it to work)

Example Code

Automatically network a color field. The following script also changes the color randomly on click

import { Behaviour, IPointerClickHandler, PointerEventData, Renderer, RoomEvents, delay, serializable, showBalloonMessage, syncField } from "@needle-tools/engine";
import { Color } from "three"

export class Networking_ClickToChangeColor extends Behaviour implements IPointerClickHandler {

    // START MARKER network color change syncField
    /** syncField does automatically send a property value when it changes */
    @syncField(Networking_ClickToChangeColor.prototype.onColorChanged)
    @serializable(Color)
    color!: Color;

    private onColorChanged() {
        // syncField will network the color as a number, so we need to convert it back to a Color when we receive it
        if (typeof this.color === "number")
            this.color = new Color(this.color);
        this.setColorToMaterials();
    }
    // END MARKER network color change syncField

    /** called when the object is clicked and does generate a random color */
    onPointerClick(_: PointerEventData) {
        const randomColor = new Color(Math.random(), Math.random(), Math.random());
        this.color = randomColor;
    }

    onEnable() {
        this.setColorToMaterials();
    }

    private setColorToMaterials() {
        const renderer = this.gameObject.getComponent(Renderer);
        if (renderer) {
            for (let i = 0; i < renderer.sharedMaterials.length; i++) {
                // we clone the material so that we don't change the original material
                // just for demonstration purposes, you can also change the original material
                const mat = renderer.sharedMaterials[i]?.clone();
                renderer.sharedMaterials[i] = mat;
                if (mat && "color" in mat)
                    mat.color = this.color;
            }
        }
        else console.warn("No renderer found", this.gameObject)
    }

}

Simple networking of a number

import { Behaviour, syncField } from "@needle-tools/engine"

export class AutoFieldSync extends Behaviour implements IPointerClickHandler {

    // Use `@syncField` to automatically network a field. 
    // You can optionally assign a method or method name to be called when the value changes
    @syncField("myValueChanged")
    mySyncedValue?: number = 1;
    
    private myValueChanged() {
       console.log("My value changed", this.mySyncedValue);
    }
    
    onPointerClick() {
       this.mySyncedValue = Math.random();
    }
}

Flatbuffers for your own components

• Using the schema compiler
• Generating a schema
• Flatbuffer in Typescript

Networking Package

Needle Engine currently uses its own networking package hosted on npm. By default if not configured differently using the Networking component Needle Engine will connect to a server running on Glitch.

It can be added to your own fastiy or express server running on any server for example by adding the following code on your server after installing the package:

import networking from "@needle-tools/needle-tiny-networking-ws";
networking.startServerFastify(fastifyApp, { endpoint: "/socket" });

The following options are available:

endpoint string	relative path to the websocket endpoint (e.g. /socket)
maxUsers number	Amount of users allowed per room
defaultUserTimeout number	Timeout length in seconds until a user is kicked from a room (if no ping is received). Defaults to 30 seconds

Networking on Glitch

When deploying your app to Glitch, we include a simple networking backend that is great for prototyping and small deployments (~15-20 people at the same time). You can later update to a bigger/better/stronger networking solution if required.

Limitations

• approx. 15-20 people maximum – afterwards the small default Glitch server instance becomes slow

Local Networking

For testing and development purposes it can be desired to run the needle engine networking package on a local server. We have prepared a repository that is setup to host the websocket package and to make that easy for you. Please follow the instructions in the linked repository:

• Local Networking Repository

Hosting your own Networking Server

You can also deploy your own networking server on e.g. google cloud. For further instructions please refer to the description found here: Local Networking Repository

If you want to use a different server for your local development and your hosted development (and the hosted server is not the same as your needle engine deployed website) then you can also enter a absolute URL in the Networking component URL field as well:

peerjs

Needle Engine Screencapture / Screensharing and Voip components use peerjs for networking audio and video.

Customizing peerjs options

• If you want to modify the default peerjs options you can call setPeerOptions(opts: PeerjsOptions) with your custom options. This can be used to modify the hosting provider in case where you host your own peerjs server.