Scratch Sockets
Have you ever had an idea so bad, so nonsense, so stupid, you just had to do it?
Because I had.
i've ported sockets to @scratch, then made a HTTP server pic.twitter.com/ALCBx3n2WG
— szymszl (@szymszl) July 22, 2019
Why not implement sockets in Scratch?
It would allow the already quite good language to be used somewhat professionally. It would allow Scratch to interface through the almighty Web, without being stuck within the confines of that little 640x480 screen!
Installing Scratch locally
Thinking these big words, I sat down and had a look at Scratch's extension system. A quick read through the documentation told me that I can't use custom extensions within official Scratch, but I can clone their repo and do stuff locally. According to the Getting Started guide, I installed Node 8 and ran:
$ mkdir scratch $ cd scratch $ git clone https://github.com/llk/scratch-gui $ git clone https://github.com/llk/scratch-vm $ cd scratch-vm $ npm install $ npm link $ npm run watch Let this run, and in another terminal: $ cd scratch/scratch-gui $ npm install $ npm link scratch-vm $ npm start Also let this run.
After looking at npm's colorful deprecation and vulnerability warnings, I remembered why I hate node.js's ecosystem. To my suprise, http://localhost:8601 presented me with a familiar UI, so I went on.
Creating extensions
After some confused grepping I found a list of extensions in
scratch-gui/.
I copied music's dict and changed some names.
After some more confused grepping and finding I found scratch-vm/src/extensions,
where I quickly dropped my own folder and an index.js file. After reading some
"documentation" and other extensions in the same folder,
I came up with this simple extension, which adds only one block:
// Core, Team, and Official extensions can `require` VM code:
// // Core, Team, and Official extensions can `require` VM code:
const ArgumentType = require('../../extension-support/argument-type');
const BlockType = require('../../extension-support/block-type');
const TargetType = require('../../extension-support/target-type');
// ...or VM dependencies:
const formatMessage = require('format-message');
// Core, Team, and Official extension classes should be registered statically with the Extension Manager.
// See: scratch-vm/src/extension-support/extension-manager.js
class PieceOfShit {
constructor (runtime) {
/**
* Store this for later communication with the Scratch VM runtime.
* If this extension is running in a sandbox then `runtime` is an async proxy object.
* @type {Runtime}
*/
this.runtime = runtime;
}
/**
* @return {object} This extension's metadata.
*/
getInfo () {
return {
// Required: the machine-readable name of this extension.
// Will be used as the extension's namespace.
id: 'pieceofshit',
// Core extensions only: override the default extension block colors.
color1: '#1590FF',
color2: '#2140FF',
name: formatMessage({
id: 'pieceofshit.categoryName',
default: 'piece of shit',
description: 'what the fuck'
}),
// Optional: URI for a block icon, to display at the edge of each block for this
// extension. Data URI OK.
blockIconURI: 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAkAAAAFCAAAAACyOJm3AAAAFklEQVQYV2P4DwMMEMgAI/+DEUIMBgAEWB7i7uidhAAAAABJRU5ErkJggg==',
// Optional: URI for an icon to be displayed in the blocks category menu.
menuIconURI: 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAkAAAAFCAAAAACyOJm3AAAAFklEQVQYV2P4DwMMEMgAI/+DEUIMBgAEWB7i7uidhAAAAABJRU5ErkJggg==',
blocks: [
{
opcode: 'alerta',
blockType: BlockType.COMMAND,
text: formatMessage({
id: 'pieceofshit.alertname',
default: 'alert([TEXT]);',
description: 'let\'s annoy the fuck out of the user'
}),
arguments: {
TEXT: {
type: ArgumentType.STRING
}
}
},
// Optional: define extension-specific menus here.
menus: {
},
translation_map: {
}
};
};
alerta(args) {
alert(args.TEXT);
};
}
module.exports = PieceOfShit;
I also had to register this extension in scratch-vm/src/extension-support/extension-manager.js
const builtinExtensions = {
// This is an example that isn't loaded with the other core blocks,
// but serves as a reference for loading core blocks as extensions.
[...]
pieceofshit: () => require('../extensions/scratch3_pieceofshit'),
};
After some random debugging, I finally had something nice:
Sockets from a browser
Note that extensions are executed client-side! Now I had to figure out how to create sockets from a browser.
After some thought I decided to make a HTTP API (the "cursed HTTP->sockets api") which runs the socket stuff according to Scratch's wishes. This is what I thought out:
/socket/[tcp|udp] -> allocates a socket and returns its ID /connect/[id]/[host]/[port]/ -> connects a socket to a selected host and port /send/[id]/[content]/ -> sends content (passed base64-encoded) through a socket /sendchar/[id]/[codepoint]/ -> sends a single byte (passed as a decimal codepoint) through a socket /recv/[id]/[buf]/ -> receives buf bytes from a socket, returns it base64-encoded /recvchar/[id]/ -> receives a single byte from a socket, returns it as a decimal codepoint /readuntil/[id]/[char]/ -> receives characters until char (byte codepoint) is received /bind/[id]/[port]/ -> binds a socket to a port and starts listening /accept/[id]/ -> returns a socket ID for an incoming connection for a listening socket /close/[id]/ -> closes socket
Note the absolute lack of error handling. It's also all GET requests. It was made at 2AM and just had to work.
This is the "final" implementation of the API in Python:
#!/usr/bin/env python3
import socket
import random
import base64
import traceback
from http.server import BaseHTTPRequestHandler, HTTPServer
sockets = {}
def btoa(b):
return base64.b64encode(b).decode().replace('/', '_').replace('+', '-').encode()
def atob(b):
return base64.b64decode(b.replace('_', '/').replace('-', '+').encode())
class RequestHandler(BaseHTTPRequestHandler):
def do_GET(self):
if self.path[-1] != '/':
self.path = self.path + '/'
self.args = self.path.split('/')[1:-1]
if len(self.args) == 0:
return self._index()
if callable(self.routes.get(self.args[0])):
try:
return self.routes[self.args[0]](self)
except Exception:
traceback.print_exc()
self.send_response(500)
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
return
self.send_response(404)
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
def _index(self):
# Send response status code
self.send_response(200)
# Send headers
self.send_header('Content-type', 'text/html')
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
# Send message back to client
message = (
'Welcome to <b>szy\'s cursed HTTP -> socket api</b>!<br>patent pending'
)
# Write content as utf-8 data
self.wfile.write(bytes(message, 'utf8'))
def _socket(self):
if self.args[1] not in ('tcp', 'udp'):
self.send_response(404)
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
return
name = str(random.randint(1000, 9999))
while name in sockets:
name = str(random.randint(1000, 9999))
sockets[name] = socket.socket(
type={'tcp': socket.SOCK_STREAM,
'udp': socket.SOCK_DGRAM}[self.args[1]]
)
self.send_response(200)
self.send_header('Content-type', 'text/plain')
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
self.wfile.write(name.encode())
def _connect(self):
if self.args[1] not in sockets:
self.send_response(404)
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
return
s = sockets[self.args[1]]
name = socket.gethostbyname(self.args[2])
s.connect((name, int(self.args[3])))
self.send_response(204)
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
def _send(self):
if self.args[1] not in sockets:
self.send_response(404)
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
return
s = sockets[self.args[1]]
s.send(atob(self.args[2]))
self.send_response(204)
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
def _recv(self):
if self.args[1] not in sockets:
self.send_response(404)
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
return
s = sockets[self.args[1]]
buf = s.recv(int(self.args[2]))
self.send_response(200)
self.send_header('Content-type', 'text/plain')
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
self.wfile.write(btoa(buf))
def _readuntil(self):
if self.args[1] not in sockets:
self.send_response(404)
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
return
s = sockets[self.args[1]]
until = int(self.args[2])
buf = bytearray()
for _ in range(1024):
char = s.recv(1)
if len(char) == 0:
break
buf += char
if char[0] == until:
break
self.send_response(200)
self.send_header('Content-type', 'text/plain')
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
self.wfile.write(btoa(buf))
def _sendchar(self):
if self.args[1] not in sockets:
self.send_response(404)
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
return
s = sockets[self.args[1]]
s.send(bytes([int(self.args[2])]))
self.send_response(204)
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
def _recvchar(self):
if self.args[1] not in sockets:
self.send_response(404)
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
return
s = sockets[self.args[1]]
char = s.recv(1)
if not char:
char = 'EOF'
else:
char = str(ord(char))
self.send_response(200)
self.send_header('Content-type', 'text/plain')
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
self.wfile.write(char.encode())
def _bind(self):
if self.args[1] not in sockets:
self.send_response(404)
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
return
s = sockets[self.args[1]]
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.settimeout(5)
s.bind(('0.0.0.0', int(self.args[2])))
s.listen()
self.send_response(204)
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
def _accept(self):
if self.args[1] not in sockets:
self.send_response(404)
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
return
s = sockets[self.args[1]]
try:
conn, addr = s.accept()
name = str(random.randint(1000, 9999))
while name in sockets:
name = str(random.randint(1000, 9999))
sockets[name] = conn
self.send_response(200)
self.send_header('Content-type', 'text/plain')
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
self.wfile.write(name.encode())
except socket.timeout:
self.send_response(204)
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
def _close(self):
if self.args[1] not in sockets:
self.send_response(404)
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
return
s = sockets[self.args[1]]
try:
s.shutdown(socket.SHUT_RDWR)
except:
pass
s.close()
del sockets[self.args[1]]
self.send_response(204)
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
routes = {
'socket': _socket,
'connect': _connect,
'send': _send,
'recv': _recv,
'readuntil': _readuntil,
'sendchar': _sendchar,
'recvchar': _recvchar,
'close': _close,
'bind': _bind,
'accept': _accept,
}
server_address = ('127.0.0.1', 8081)
httpd = HTTPServer(server_address, RequestHandler)
print('server up...')
httpd.serve_forever()
It's not perfect, but it worked. Also, here is the extension code, to pair:
// Core, Team, and Official extensions can `require` VM code:
// // Core, Team, and Official extensions can `require` VM code:
const ArgumentType = require('../../extension-support/argument-type');
const BlockType = require('../../extension-support/block-type');
const TargetType = require('../../extension-support/target-type')
const nets = require('nets');;
// ...or VM dependencies:
const formatMessage = require('format-message');
// Core, Team, and Official extension classes should be registered statically with the Extension Manager.
// See: scratch-vm/src/extension-support/extension-manager.js
class PieceOfShit {
constructor (runtime) {
/**
* Store this for later communication with the Scratch VM runtime.
* If this extension is running in a sandbox then `runtime` is an async proxy object.
* @type {Runtime}
*/
this.runtime = runtime;
this.socketserver = "http://localhost:8081";
}
/**
* @return {object} This extension's metadata.
*/
getInfo () {
return {
// Required: the machine-readable name of this extension.
// Will be used as the extension's namespace.
id: 'pieceofshit',
// Core extensions only: override the default extension block colors.
color1: '#1590FF',
color2: '#2140FF',
name: formatMessage({
id: 'pieceofshit.categoryName',
default: 'piece of shit',
description: 'what the fuck'
}),
// Optional: URI for a block icon, to display at the edge of each block for this
// extension. Data URI OK.
blockIconURI: 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAkAAAAFCAAAAACyOJm3AAAAFklEQVQYV2P4DwMMEMgAI/+DEUIMBgAEWB7i7uidhAAAAABJRU5ErkJggg==',
// Optional: URI for an icon to be displayed in the blocks category menu.
menuIconURI: 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAkAAAAFCAAAAACyOJm3AAAAFklEQVQYV2P4DwMMEMgAI/+DEUIMBgAEWB7i7uidhAAAAABJRU5ErkJggg==',
// Required: the list of blocks implemented by this extension,
// in the order intended for display.
blocks: [
{
opcode: 'alerta',
blockType: BlockType.COMMAND,
text: formatMessage({
id: 'pieceofshit.alertname',
default: 'alert([TEXT]);',
description: 'let\'s annoy the fuck out of the user'
}),
arguments: {
TEXT: {
type: ArgumentType.STRING
}
}
},
{
opcode: 'chr',
blockType: BlockType.REPORTER,
text: formatMessage({
id: 'pieceofshit.chr',
default: 'chr([NUMBER])'
}),
arguments: {
NUMBER: {
type: ArgumentType.NUMBER,
}
}
},
{
opcode: 'ord',
blockType: BlockType.REPORTER,
text: formatMessage({
id: 'pieceofshit.ord',
default: 'ord([CHAR])'
}),
arguments: {
CHAR: {
type: ArgumentType.STRING,
}
}
},
{
opcode: 'setSocketServer',
blockType: BlockType.COMMAND,
text: formatMessage({
id: 'pieceofshit.socketserver',
default: 'Use cursed HTTP -> socket api @ [TEXT]',
description: 'well'
}),
arguments: {
TEXT: {
type: ArgumentType.STRING,
defaultValue: "http://localhost:8081"
}
}
},
{
opcode: 'getSocketServer',
blockType: BlockType.REPORTER,
text: formatMessage({
id: 'pieceofshit.getsocketserver',
default: 'used cursed HTTP -> socket api',
description: 'well'
}),
},
{
opcode: 'openSocket',
blockType: BlockType.REPORTER,
text: formatMessage({
id: 'pieceofshit.openSocket',
default: 'open a new socket type [SOCKTYPE]'
}),
arguments: {
SOCKTYPE: {
type: ArgumentType.NUMBER,
menu: 'SOCKETTYPE',
defaultValue: 1
}
}
},
{
opcode: 'connect',
blockType: BlockType.COMMAND,
text: formatMessage({
id: 'pieceofshit.connect',
default: 'Connect socket [SOCKET] to [ADDR]:[PORT]',
description: 'well'
}),
arguments: {
SOCKET: {
type: ArgumentType.STRING,
},
ADDR: {
type: ArgumentType.STRING,
},
PORT: {
type: ArgumentType.NUMBER,
},
}
},
{
opcode: 'accept',
blockType: BlockType.REPORTER,
text: formatMessage({
id: 'pieceofshit.accept',
default: 'accept a connection from socket [SOCKET] or return empty'
}),
arguments: {
SOCKET: {
type: ArgumentType.STRING,
},
}
},
{
opcode: 'bind',
blockType: BlockType.COMMAND,
text: formatMessage({
id: 'pieceofshit.bind',
default: 'Bind socket [SOCKET] to port [PORT]',
description: 'well'
}),
arguments: {
SOCKET: {
type: ArgumentType.STRING,
},
PORT: {
type: ArgumentType.NUMBER,
},
}
},
/*{
opcode: 'isopen',
blockType: BlockType.BOOLEAN,
text: formatMessage({
id: 'pieceofshit.isopen',
default: 'socket [SOCKET] is open',
description: 'well'
}),
arguments: {
SOCKET: {
type: ArgumentType.STRING,
},
}
},*/
{
opcode: 'send',
blockType: BlockType.COMMAND,
text: formatMessage({
id: 'pieceofshit.send',
default: 'Send [TEXT] to socket [SOCKET]',
description: 'well'
}),
arguments: {
TEXT: {
type: ArgumentType.STRING,
},
SOCKET: {
type: ArgumentType.STRING,
},
}
},
{
opcode: 'recv',
blockType: BlockType.REPORTER,
text: formatMessage({
id: 'pieceofshit.recv',
default: 'Receive [BUF] chars from [SOCKET]'
}),
arguments: {
BUF: {
type: ArgumentType.NUMBER,
defaultValue: 1
},
SOCKET: {
type: ArgumentType.STRING,
}
}
},
{
opcode: 'readuntil',
blockType: BlockType.REPORTER,
text: formatMessage({
id: 'pieceofshit.readuntil',
default: 'Read from [SOCKET] until character [CHAR]'
}),
arguments: {
CHAR: {
type: ArgumentType.NUMBER,
defaultValue: 10
},
SOCKET: {
type: ArgumentType.STRING,
}
}
},
{
opcode: 'sendchar',
blockType: BlockType.COMMAND,
text: formatMessage({
id: 'pieceofshit.sendchar',
default: 'Send character [CHAR] to socket [SOCKET]',
description: 'well'
}),
arguments: {
CHAR: {
type: ArgumentType.NUMBER,
},
SOCKET: {
type: ArgumentType.STRING,
},
}
},
{
opcode: 'recvchar',
blockType: BlockType.REPORTER,
text: formatMessage({
id: 'pieceofshit.char',
default: 'Receive one character from [SOCKET]'
}),
arguments: {
SOCKET: {
type: ArgumentType.STRING,
}
}
},
{
opcode: 'close',
blockType: BlockType.COMMAND,
text: formatMessage({
id: 'pieceofshit.close',
default: 'Close socket [SOCKET]',
description: 'well'
}),
arguments: {
SOCKET: {
type: ArgumentType.STRING,
},
}
},
],
menus: {
// Required: an identifier for this menu, unique within this extension.
SOCKETTYPE: [
// Static menu: list items which should appear in the menu.
{
// Required: the value of the menu item when it is chosen.
value: 1,
// Optional: the human-readable label for this item.
// Use `value` as the text if this is absent.
text: formatMessage({
id: 'SOCKETTYPE_TCP',
default: 'TCP',
description: 'TCP'
})
},
// The simplest form of a list item is a string which will be used as
// both value and text.
{
// Required: the value of the menu item when it is chosen.
value: 2,
// Optional: the human-readable label for this item.
// Use `value` as the text if this is absent.
text: formatMessage({
id: 'SOCKETTYPE_UDP',
default: 'UDP',
description: 'UDP'
})
},
],
},
// Optional: translations (UNSTABLE - NOT YET SUPPORTED)
translation_map: {
}
};
};
alerta(args) {
alert(args.TEXT);
};
chr(args) {
return String.fromCharCode(args.NUMBER);
}
ord(args) {
return args.CHAR.charCodeAt(0);
}
openSocket(args) {
stype = {1: 'tcp', 2: 'udp'}[args.SOCKTYPE];
console.log(`Opening socket, of type ${stype}`);
var xhttp = new XMLHttpRequest();
xhttp.open("GET", this.socketserver+'/socket/'+stype, false)
xhttp.send();
return xhttp.response;
/*return new Promise(resolve => {
nets({
url: this.socketserver+'/socket/'+stype
}, (err, res, body) {
if (err) {
log.warn(err);
return resolve();
}
return resolve(body);
}
)
});*/
};
setSocketServer(args) {
this.socketserver = args.TEXT
}
getSocketServer() {
return this.socketserver
}
connect(args) {
var xhttp = new XMLHttpRequest();
xhttp.open(
"GET",
this.socketserver+'/connect/'+args.SOCKET+'/'+args.ADDR+'/'+args.PORT,
false)
xhttp.send();
}
accept(args) {
var xhttp = new XMLHttpRequest();
xhttp.open(
"GET",
this.socketserver+'/accept/'+args.SOCKET+'/',
false)
xhttp.send();
return xhttp.response;
}
bind(args) {
var xhttp = new XMLHttpRequest();
xhttp.open(
"GET",
this.socketserver+'/bind/'+args.SOCKET+'/'+args.PORT,
false)
xhttp.send();
}
isopen(args) {
var xhttp = new XMLHttpRequest();
xhttp.open(
"GET",
this.socketserver+'/isopen/'+args.SOCKET+'/',
false)
xhttp.send();
return (xhttp.response === "true") ? true : false;
}
send(args) {
var xhttp = new XMLHttpRequest();
xhttp.open(
"GET",
this.socketserver+'/send/'+args.SOCKET+'/'+btoa(args.TEXT).replace('/', '_').replace('+', '-'),
false)
xhttp.send();
}
recv(args) {
var xhttp = new XMLHttpRequest();
xhttp.open(
"GET",
this.socketserver+'/recv/'+args.SOCKET+'/'+args.BUF,
false)
xhttp.send();
return atob(xhttp.response.replace('_', '/').replace('-', '+'));
}
readuntil(args) {
var xhttp = new XMLHttpRequest();
xhttp.open(
"GET",
this.socketserver+'/readuntil/'+args.SOCKET+'/'+args.CHAR,
false)
xhttp.send();
return atob(xhttp.response.replace('_', '/').replace('-', '+'));
}
sendchar(args) {
var xhttp = new XMLHttpRequest();
xhttp.open(
"GET",
this.socketserver+'/sendchar/'+args.SOCKET+'/'+args.CHAR,
false)
xhttp.send();
}
recvchar(args) {
var xhttp = new XMLHttpRequest();
xhttp.open(
"GET",
this.socketserver+'/recvchar/'+args.SOCKET,
false)
xhttp.send();
return xhttp.response;
}
close(args) {
var xhttp = new XMLHttpRequest();
xhttp.open(
"GET",
this.socketserver+'/close/'+args.SOCKET,
false)
xhttp.send();
}
}
module.exports = PieceOfShit;
It could definitely be improved, such as to use Promises for the requests,
all of this is just hacked together by a person who last did javascript when document.write() was all the rage.
It works though.
To finish on something, here is the code for scratchttpd:
Thanks for reading, creating this little abomination of a project was definitely fun!