A request module for slope-lang supporting http(s), gopher, and gemini

slope-lang module gemini gopher http titan

sloum cba02d4a20 Updates to using namespaces		2 weeks ago
README.md	Updates to using namespaces	2 weeks ago
main.slo	Updates to using namespaces	2 weeks ago
module.json	Updates to using namespaces	2 weeks ago

README.md

request

A request library supporting multiple protocols. At present: http(s), gopher, and gemini. In all cases default ports will be used unless a port is given as a part of the url. If a scheme is not included in the url passed to any of the main procedures a scheme will be added based on the procedure (gopher for gopher, gemini for gemini, http for http; https will never default, so it must be included in the url).

There is a global setting for connection timeout that is, by default, set to 20 (seconds). To change this: (request::set-timeout [number]).

http(s)

At present only GET and POST requests are supported (see (usage request::http-POST) or (usage request::http-GET) for further details).

(request::http-GET "https://slope.colorfield.space")

; Returns:
; (
;   ("status" "200")
;   ("version" "HTTP/1.1")
;   ("headers" (
;     ("Date" "Wed, 18 May 2022 22:54:52 GMT")
;     ("Server" "Apache/2.4.29 (Ubuntu)")
;     ("Last-Modified" "Sun, 17 Apr 2022 05:51:49 GMT")
;     ("ETag" ""25b7-5dcd33d6792cf"") ("Accept-Ranges" "bytes")
;     ("Content-Length" "9655")
;     ("Vary" "Accept-Encoding")
;     ("Content-Type" "text/html")))
;   ("body" "<!DOCTYPE html>\n<html lang="en" idr="ltr">\n [...]

gopher

Requests the given resource and returns it as a string. This will get funky with telnet and query based links (which do not return files in the regular sense).

(request::gopher "gopher://colorfield.space")

gemini

Requests the given gemini resource and returns an associative list. Response type 3 (redirect) will be automatically followed. All other response types will return the given associative list. As such response type 1x responses do not function interactively as is (you will have to read the response, query the user, and make a new request with the given querystring).

(request::gemini "gemini://gemlog.blue")

; Returns:
; (
;   ("status" "20")
;   ("meta" "text/gemini")
;   ("body" "..."))

spartan

Requests the given spartan resource and returns an associative list. Response type 3 (redirect) will be automatically followed. All other response types will return the given associative list.

(request::spartan "spartan://tilde.cafe/~hedy/")

; Returns:
; (
;   ("status" "2")
;   ("meta" "text/gemini; lang=en; charset=utf-8")
;   ("body" "..."))

helpers

A few helpers are provided to make certain kinds of requests easier:

(request::urlencode "Hello, world!")
; Returns: "Hello%2C%20world%21"

(request::urlencode [["key1" "a value!"] ["key 2" "Yay!!"]])
; Returns: "key1=a%20value%21&key%202=Yay%21%21"

(request::urldecode "Hello%2C%20world%21")
; Returns: "Hello, world!"

(request::urldecode "key1=a%20value%21&key%202=Yay%21%21")
; Returns: (("key1" "a value!") ("key 2" "Yay!!"))

(request::escape-html "This & 'that' are > \"that\" or this")
; Returns: "This &amp; &#39;that&#39; are &gt; &quot;that&quot; or this"

(request::unescape-html "This &amp; &#39;that&#39; are &gt; &quot;that&quot; or this")
; Returns: "This & 'that' are > "that" or this"

Do note that because slope does not opperate on bytes multibyte characters will not encode as expected for utf-8 documents. Instead of multibyte percent encoding a single byte percent encoding will be created based on a unicode code point (rune). This can still be workable for http POST requests if you set the encoding to something like Windows-1252, which uses single byte code points that mostly line up with the utf-8 runes in use in slope. In any case, these two helpers are mostly suited to ascii text and should not be rellied upon for non-hobby code (ie. not for use in production).

generic

If you just want to retrieve the response body from a given request and don't care about any other details or want to work with a bunch of protocols but always call the same proc you can use:

(request::fetch "gopher://colorfield.space")
(request::fetch "gemini://rawtext.club")
(request::fetch "http://slope.colorfield.space")
(request::fetch "https://sloum.colorfield.space")
(request::fetch "spartan://tilde.cafe/")

; In all cases will return the response body
; as a string or `#f`. The response body may
; be an empty string, particularly for an error
; response from many of the available protocols.
; An exception will be thrown if it was not
; possible to connect with the host at all.