switch to room list switch to menu My folders
Go to page: First ... 28 29 30 31 [32] 33 34 35 36 ... Last
[#] Mon Jul 06 2009 10:21:49 EDT from Ford II @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

well, I thought about that.
I'm not done with the throttling yet, but when you type something in ssh, it doesn't wait a second, it forces a hit right away. The problem is in data coming back, there's no way to trigger a hit.
so I figure the best I can do is when I see data coming back, keep the poll frequency high until it quiets down.
I still have a lot of screwing around to do, but the first go worked surprisingly well.

[#] Mon Jul 06 2009 10:33:45 EDT from Peter Pulse @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

Maybe if there's no data waiting to go back, the passive daemon shouldn't send the http response right away, but wait a short time (50-200 milliseconds??) to see if something comes back in response to what it just sent. Or, better yet, use multiple connections. One http connection waits around until data is available to bring back (or until some set timeout period after which you give up and send a response with no data.. and then another http connection is made which again waits). Then you can make connections to send data at will...

[#] Mon Jul 06 2009 11:10:24 EDT from Ford II @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

The only problem I see there is if the http request (waiting for responses) times out before some unsolicited data comes back.
I suppose the best way to do it depends on what kind of traffic you're going to be sending back and forth.
If you're tailing a log, you'll never have any data triggering from the client, so that response http request would have to sit there indefinetly waiting for something to be added to the log file you're tailing.
The problem with doing what I'm doing now (polling every 1-5 seconds) is that it looks suspicious in the web proxy logs which could give you away.
But I figure google mail is constantly making xmlhttprequests to update its status and the mail page, so I'm hoping I'll get lost in the noise.

[#] Mon Jul 06 2009 11:34:41 EDT from Peter Pulse @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

I'm not suggesting waiting forever. I'm suggesting two scenarios. The first, where you are only dealing with one http request at a time, like you have now.. I am suggesting that when the passive end gets the http request asking it to transmit some data.. if it then checks its buffer and finds out that there is nothing to send back in the response, it doesn't give up and send back an empty response.. but instead waits just a very short time to see if something comes in that it can send back. Since the next http request is not coming for 1 second, it might as well wait around on this http request for up to even 500 milliseconds since if it's an interactive session, the echo will probably come back much more quickly (within 20-100 ms) and it would be a shame to immediately give up and send that request back empty. If you are going to send it back empty, there is nothing to lose by waiting a little bit and a lot to be gained.

The second thing I suggest is to juggle multiple sockets.. and it doesn't have to be complicated, two simultaneous http requests is enough. The first http connection, you take the data to be sent, if any.. send it.. then you wait around waiting for some data to be sent back. And you can wait a while, since the proxy isn't going to time out very fast, you could probably wait at least a minute. Then you have two choices on that request.. you can wait until some data comes for you to send back, then you send a response and close it and wait for the next request. Or you can keep it open and do server push, which has been around for a while and the proxy should deal with it. Meanwhile, you also accept new http requests to send data, which you accept immediately and send.

If you were trying to do all this in one process it would make it a lot more complicated, but since you are already doing a daemon and cgi program thing, it's actually not that complicated because the cgi program worries about how long it is going to wait. The active end does become more complicated though.

[#] Mon Jul 06 2009 12:00:33 EDT from Ford II @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

Ahhh I misunderstood.
Okay, putting a select on the server side to wait 500 ms is a good idea.
Although consider an interactive session:
I'm in ssh and I'm typing a lot of character, from putty, from what I've seen so far, each character you type ends up being a request, because I type so slow two keystrokes never get bunched together in one request.
So if I'm in joe and I type a lot of stuff, it actually looks completely normal, except the last chacter doesn't show up until after a second pause.
So I can try waiting 500 ms and see if that helps.
I'm also thinking the throttling down will help if I let it go to 5 minutes between hits so it doesn't show up on the proxy log radar.
Since what *I'll* be doing is ssh, all of the activity will be pretty much driven by my typing, so I don't need great response from the server. Unless I'm doing something to trigger action.

Never heard of server push. Vus is dus?

The other thing I'm going to work in, which should be real simple since most of the infrastructure is there, is a reverse connection.
daemon2 will listen on yet another port and create a connection backwards, so you can get in to the firewall the other way. This will really make people mad.
As it is, both daemons are state machines, they can each handle multiple connections on both sides, so adding more stuff like that shouldn't be too hard.

[#] Mon Jul 06 2009 12:05:17 EDT from Ford II @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

Well, I'm connectied here via a real web proxy from my desk at work.
It's like being on a really laggy connection but it works.
Better than no connectivity at all.
I'm going to try a web proxy instead of ssh and see what it's like.

[#] Mon Jul 06 2009 12:10:03 EDT from Peter Pulse @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

completely normal, except the last chacter doesn't show up until after

a second pause.
So I can try waiting 500 ms and see if that helps.

Yea exactly, that character (or cursor movement or whatever) probably comes back within a few milliseconds but the response was already sent back empty.
If you waited around a few milliseconds the latency would be much less. This way you are using that time between your keystrokes.

[#] Mon Jul 06 2009 13:04:36 EDT from IGnatius T Foobar @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

Okay, putting a select on the server side to wait 500 ms is a good

Eventually you're going to end up reimplementing the Nagle algorithm.

Or you could get it for free by tunnelling a virtual network interface over it and then letting the operating system layer its own Nagle on top.

Other implementations of this seem to be fond of using SOCKS as the layer immediately above the HTTP transport, but I don't think that gives you any performance benefits other than convenience.

[#] Mon Jul 06 2009 14:27:56 EDT from Ford II @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

Less library my way. :-)
Although if I'm going to make an https connection, I'll have to get all that tls stuff working. :-(

I'm next going to try the citadel client, because I figure that is message based I won't notice any lag.

Any chance the cit client builds on cygwin?

[#] Mon Jul 06 2009 14:54:58 EDT from IGnatius T Foobar @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

It definitely used to, but we haven't tried it in a while.

[#] Mon Jul 06 2009 16:14:40 EDT from Ford II @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

I'll give it a go.

[#] Mon Jul 06 2009 16:47:11 EDT from fleeb @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

I found this comment in some code here:

_alloca is vile, evil, and leaks worse than your wife's titty after she's given birth.

It... certainly stands out.

[#] Tue Jul 07 2009 07:01:45 EDT from Ford II @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

So I can try waiting 500 ms and see if that helps.

Yea exactly, that character (or cursor movement or whatever) probably

works nicely. thanks.

[#] Tue Jul 07 2009 07:05:43 EDT from Ford II @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

joe and bash respond within 200ms, uncensored does not. :-
But the response is still verynice.

[#] Tue Jul 07 2009 11:39:05 EDT from Peter Pulse @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

Cool. To answer your question from yesterday.. server push is where you don't close the http connection at the server.. you send the header and then you just keep sending data a little at a time as it becomes available. When used with a browser, there are obviously some rules that must be followed if you want the browser to for example update an image to allow you to display a live video feed from a camera (by sending a stream of jpg image frames) then you must send the right mime types (eg multipart/x-mixed-replace), and if you are trying to implement let's say a chat program then you have to send the right html to get the browser to draw .. but if you are not talking to a browser, you have a lot more lattitude. For example if you send back a document with a text/plain mime type or some kind of binary mime type, you can effectively do it forever or for a very long time, so long as you don't pause so long that something times out. The proxy has to allow it.. after all, you might just be downloading something very big on a very slow connection, it could take hours right? But the problem for CGI programs is that the server doesn't normally give you the real socket descriptor, it buffers your CGI output so that it can add its own headers (like the size header).. but you can get around this by using the nph (non-parsed header) mode.. in apache you just name your cgi program nph-mycgi.cgi and it will give you the real socket and sit back and let you do what you want. The burden is then on you to send the miniumum valid saet of http headers and not just content-type, since the server isn't helping you anymore. It isn't that hard, I have done it a bunch of times when I had CGI scripts that are doing some big long task that I wanted to follow.. like the log file output you mentioned yesterday. For example at I wrote a script that would synchronize pre-production over to production.. and this was more than a simple copying because like Harris there were replacements that had to be done, also database stuff that had to be copied. Production was in Virginia on multiple machines behind a firewall and preproduction was on one machine in my office... with an SSH pipe set up to do an Oracle db link. Anyway, not to be distracted by nostalgia.. but so anyway I wrote this web interface so that the production (html) people could choose what they wanted to put in production and then make it go over, and see as it went over any details of what was going on or going wrong.

[#] Tue Jul 07 2009 17:23:46 EDT from Ford II @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

Interesting. Also one of the things I want to avoid is being spotted on the web proxy.
So my connections are short and go away.
I've got the throttling thing working, and it seems you can sit at a bash prompt in ssh and send no data either way, so the poll times get farther and farther apart.
It seems citadel sends data though :-(

[#] Wed Jul 08 2009 09:51:54 EDT from IGnatius T Foobar @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

Yes, at the very least it sends a keepalive every 30 seconds. You may also find that with other protocols (telnet ssh etc) you will probably find that you're either going to see keepalives or that one end or the other sometimes drops the session for lack of them.

[#] Wed Jul 08 2009 11:46:23 EDT from Ford II @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

I left an ssh sitting all night, didn't send a single packet. That's the way it should be. I realize it's not realistic, but it would be nice if things just worked like that.

[#] Wed Jul 08 2009 16:35:42 EDT from dothebart @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

ssh has an optional feature to send a ping so it closes on disconnects.

[#] Thu Jul 09 2009 18:07:19 EDT from dothebart @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

mono seems to win over java.

I guess he makes a good point.

microsofts plan to distract developers from java finaly caught on...

the gpl'ing of java was much to late...

Go to page: First ... 28 29 30 31 [32] 33 34 35 36 ... Last