[gopher] gopher++ (gopher1) protocol

[Kim Holviala]

On 11.1.2010 12:03, Mateusz Viste wrote:

>> So yes, a gopher++ client sends an empty CRLF when it's done with
>> the headers. A gopher++ server is required to read the gopher0
>> selector and wait for some predefined time for the extra headers
>> (0.01 seconds?) and if the extra headers don't come it's expected
>> to handle the request as gopher0.
>
> That's not a way to go. What if the TCP packet containing your second
> line got lost? The TCP retransmission mechanism won't resend it in
> those 0.01 second.

First of all - I was _going_ to test this all out live with actual
clients and actual live internet connections before even talking about
it. But since I broke my server software so badly rewriting it (which I
don't regret :-) it's going to take some time before I can actually do
it. Might have been stupid, but I decided to just post the ideas untested.

I was going to address that packet loosing thing too. A TCP packet is
generally at least 1K in size. Why would a gopher++ client purposefully
split the selector and the first header line into two TCP packets? They
fit into one packet just fine, and the rest of the headers can come in
separate TCP packets as after the first extra header the server will
read headers until that empty line.

> As I wrote before, there *must* be some kind of a flag in the very
> first line of the Gopher request, which would indicate to the server
> wheter it has to wait for other lines to come or not.

That breaks gopher0 which I was not willing to do. Extra lines don't.

>> the client could request a special selector (say, "server-status")
>>  which would then contain machine-readable information about the
>> server.
>
> What about a CAPABILITIES selector? That's already used in NNTP and
> POP3, and probably in some other protocols...

True. I just took the "server-status" from Apache/LigHTTPD... I think
CAPABILITIES is actually a better one as server-status doesn't really do
the same thing....

What about for format for CAPABILITIES? Mime-style "Key: Value"?
Basically the most important thing there would be the gopher version
supported (0, + or 1).

>> If the server cannot transcode the resource then the resource is
>> sent as is.
>
> That's not what you wrote before: "the server MUST send its reply
> using the format the client requested.".

I never said the doc would be coherent :-). I was way too tired when I
was writing it - I just wanted to spit the first version out as soon as
possible and fix it later (I'm not much of a writer).

[...]

"If the client does not send the Accept header, if the server doesn't
recognize the content-type client requested, or if the client requests a
content-type of "application/octet-stream", the server must serve the
resource in its original format, or format it thinks the client expects. "

Basically, that gives the server a backdoor to just dump the resource as
is to the client, just like in gopher0.

> Anyway, the whole transcoding idea seems odd to me.

The reason was to make the client as simple as possible. It's easy to 
add support to new formats in the server - it's really freaking hard to 
add support in clients. Witness PNG in HTML - it took about nearly 10 
years to get accepted even though it was clearly a superior format to 
GIF - all because client support wasn't there (MsIE).

> The client already know what to expect, as that's specified in the
> gopher URL by the gopher type...

Yeah, but that gopher type never gets to the server so the _server_
doesn't know what to give to the client.

Consider the selector "/test". What's a server to do? Serving out an MP3
file called "test" is a perfectly valid reply. Serving out a text file
called "test" is just as good.

> Plus, you're limiting the protocol to some specific file format,
> which is plainly wrong.

I actually just took the lowest common denominators. And I'm not
limiting anything - a client can always send out an "Accept:
application/octet-stream" to get the original data unmodified.

> About the "Accept-Charset" header: It seems useless to me... The
> server won't be able to transcode a given 8bit charset into another
> one, as that's simply impossible.

What's wrong with iconv(3)? Of course the server has to know the
originating charset to do the conversion, but that's a _server_ problem,
not a client problem. Right now the server dumps out stuff and then the
client has to figure out what it got...

> Why not simply stating, that any Gopher++ document *has* to be
> provided in UTF-8?

Because that's not compatible with gopher0.




- Kim