Language:
switch to room list switch to menu My folders
Go to page: First ... 39 40 41 42 [43] 44 45 46 47 ... Last
[#] Tue Nov 10 2009 12:26:35 EST from IGnatius T Foobar @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

Nope. One thread/process gets woken up, all the others remain in a
blocking state.

I think that's called "wake one" ? Is that the default behavior on Linux now? It hasn't always been that way, has it?

(I'm guessing LS will have the most reliable knowledge of this)

[#] Tue Nov 10 2009 12:32:25 EST from IGnatius T Foobar @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

Not every thread. One thread at a time. If you look closely, you
will see that the section of code which calls select() is wrapped in a

mutex so that only one thread does it at a time.

Or at least we used to. Someone (possibly you?) may have changed it.

[#] Tue Nov 10 2009 12:40:07 EST from davew @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

 

Tue Nov 10 2009 12:25:50 PM EST from IGnatius T Foobar @ Uncensored
If there is no activity then every thread will, in turn, construct a
list of ALL the client socket fd's (there is a mutex around these as
they are in contexts). Then each thread will add ALL of the master
socket fd's to its list and then each thread will call select().

Not every thread. One thread at a time. If you look closely, you will see that the section of code which calls select() is wrapped in a mutex so that only one thread does it at a time.

NO, IT IS NOT.

If it were the probable mutext would be S_SESSION_TABLE which other threads use when parsing the list of contexts which would cause deadlock or at least ver slow client processing.

 



[#] Tue Nov 10 2009 12:41:28 EST from davew @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

 

Tue Nov 10 2009 12:32:25 PM EST from IGnatius T Foobar @ Uncensored
mutex so that only one thread does it at a time.

Or at least we used to. Someone (possibly you?) may have changed it.

Hehe, well now that is a possibility but I'll need to check the svn log.

 



[#] Tue Nov 10 2009 13:06:54 EST from davew @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

 

Tue Nov 10 2009 12:41:28 PM EST from davew @ Uncensored

 

Tue Nov 10 2009 12:32:25 PM EST from IGnatius T Foobar @ Uncensored
mutex so that only one thread does it at a time.

Or at least we used to. Someone (possibly you?) may have changed it.

Hehe, well now that is a possibility but I'll need to check the svn log.



IG, you are indeed correct, at one time it was in a mutex, way back before my time.

I'm now trawling the log to see if I can find out why that mutex was removed.

 



[#] Tue Nov 10 2009 16:32:41 EST from Ford II @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

I think that's called "wake one" ? Is that the default behavior on

Linux now? It hasn't always been that way, has it?

(I'm guessing LS will have the most reliable knowledge of this)

I've played with lots of linuxes, aix solaris and I think even hpux and I've never seen it operate otherwise. If it's different historically, then it goes way back.

[#] Tue Nov 10 2009 16:33:59 EST from Ford II @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

something somebody said doesn't make sense.
all threads select at the same time (Still up for debate I guess) but all threads lead to accept.
That can't be. There are still active sessions, no?

[#] Tue Nov 10 2009 16:41:08 EST from Ford II @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

First thread to accept() on one particular master socket wins, the
others then fail the accept() on that socket and continue itterating
the loop. Winning thread does connection setup and goes back to
rebuild its fd set.

This also can't be. the others will block. Unless you're using non blocking sockets at which point it will return immediately with eagain. I'm pretty sure you're not using non blocking sockets.
If you are unintentionally blockingthreads on accept calls, I imagine performance would really be sucking because everybody would be handled on a very few thread.
But wait, there's only one socket to call accept on, the listen socket. You're sucking me into your weird world here.

[#] Tue Nov 10 2009 16:48:12 EST from Ford II @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

I'm trying to remember what I saw yesterday (which was a few months old svn checkout) and I do remember something where after the select, it checks the listen socket first and deals with it, but I don't remember if it falls through after that or not.
I do remember the select be locked in a mutex, but it won't help if the fdset list is figured out outside the mutex. If it is made outside the mutex, you probably have a problem of multiple threads processing things that have already been dealt with by other threads (under load anyway).
It probably doesn't cause any serious harm because as IG said each client processing context is locked by a mutex.
So at worst you're selecting pointlessly or at worst tying up a thread blocking on accept where you don't mean to.
and if you have any regularly schedule jobs, they will clean up the mess and make the problem less obiovus.
Or, you know what? webcit, I'm guessing it doesnt persist connections to the server? If it does not, then every web hit causes some thread to stop blocking on accept, so you may never actually see a problem, except at dead times, and there probably aren't many, and if there are, the system's idle so nobody notices.

[#] Tue Nov 10 2009 17:22:28 EST from davew @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

Ford,

The description I gave is correct, maybe a little confusing but correct.

Every thread select() with the same fd's.

Every thread heads towards accept()

sockets are non blocking until after accept().

all threads iterate list of fd's looking for master sockets and call accept() on first one they find.

If that accept handles the one and only pending connection on that socket then any other accept() on that socket will fail and the thread that did it will continue iterating. Other wise another pending connection on that socket will get handled.

We have more than 1 master socket (cit proto, pop3, smtp, imap......)

After a thread has iterated the master sockets and none of them accept() it falls through to deal with client sockets.

There is currently not, DEFINATELY NOT any mutex or locking around select() and there has not been since 2004.

Ultimately all the threads are let loos from select() and picked out one at a time thereafter either by accept() and done or by a mutex around the client sockets which allows only one thread to pick one client at a time.

CtdlThreadSelect() is a wrapper around select() but does not lock the select. The wrapper is there to colate information about how long a thread is idle in select.

 



[#] Tue Nov 10 2009 17:28:03 EST from davew @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

 

Tue Nov 10 2009 04:48:12 PM EST from Ford II @ Uncensored
Or, you know what? webcit, I'm guessing it doesnt persist connections to the server? If it does not, then every web hit causes some thread to stop blocking on accept, so you may never actually see a problem, except at dead times, and there probably aren't many, and if there are, the system's idle so nobody notices.

The situation that brought all this about was one I noticed on my test box.

Some numpty was trying to get in via pop3 and was starting lots of connections very quickly.

The upshot was that all my threads were busy handling accept() all the time.

The active session I had from before hand was useless since no thread got chance to service it.

The new connections he was creating were only getting the greating message and nothing else.

Ultimately it created DOS.



[#] Tue Nov 10 2009 18:11:55 EST from dothebart @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

dave, maybe msmtp can help to re-check your situation?



[#] Tue Nov 10 2009 18:51:36 EST from LoanShark @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

Really? Doesn't that go against the generally understood behavior of

the "thundering herd" problem? O'

Yes and no. 5 or 10 years ago, doing it this way was a problem. But on modern Linux, it's not a problem, Linus fixed in in the kernel, and he had some rather choice words for the Apache guys who had hacked around the problem with flocking and whatnot, because they had masked the problem and it would have been fixed sooner if they hadn't masked it.

[#] Tue Nov 10 2009 18:55:05 EST from LoanShark @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

So if there is no activity EVERY thread will eventually end up in
select with the exact same fd sets. Yes I know Citadel times out its

I thought Citadel had select() wrapped in a mutex, but I haven't looked at the code in several years.

[#] Tue Nov 10 2009 18:59:07 EST from LoanShark @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

I think that's called "wake one" ? Is that the default behavior on

Linux now? It hasn't always been that way, has it?

Linux' accept() has not always been wake-one, but it has been wake-one for so many years now that I would say it's safe to remove any vestigial mutex code. Other OS'es are inferior and are not of concern.

[#] Tue Nov 10 2009 19:03:03 EST from LoanShark @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

I've played with lots of linuxes, aix solaris and I think even hpux
and I've never seen it operate otherwise. If it's different
historically, then it goes way back.

says ford, referring to accept().

when accept() is not wake-one, you might not even notice, because the application layer semantics are still the same. accept() never returns anything to the clients that "lose." It's just that, on early Linux and probably a few other similarly buggy Unix kernels, accept() would wake up *inside the kernel*, notice that another process had "won", and then go back to sleep again -- all *inside the kernel.* So from the application layer, you have no way of knowing this is happening, unless it happens often enough and with enough processes that you actually notice the CPU load spiking in "top" or something, or your benchmarks show a problem. Assuming you're actually *doing* any benchmarking :)

[#] Tue Nov 10 2009 19:04:15 EST from LoanShark @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

This also can't be. the others will block. Unless you're using non
blocking sockets at which point it will return immediately with eagain.


(referring to accept) this is correct.

[#] Wed Nov 11 2009 02:28:26 EST from dothebart @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

 

Di Nov 10 2009 18:11:55 EST von dothebart @ Uncensored

dave, maybe msmtp can help to re-check your situation?



hm, it was mstone I was talking about. _this_ definitely is one of my tripple-m's, IG.



[#] Wed Nov 11 2009 05:30:48 EST from dothebart @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

lets rewrite citadel in go: http://golang.org/



[#] Wed Nov 11 2009 09:38:04 EST from IGnatius T Foobar @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

Linux' accept() has not always been wake-one, but it has been wake-one

for so many years now that I would say it's safe to remove any
vestigial mutex code. Other OS'es are inferior and are not of concern.


Thanks LS, that's exactly what I needed to hear.

davew: this means that the removal of the mutex from the Citadel server a couple of years ago was the correct course of action. It also means that we can do the same in WebCit (and I may begin experimenting with that).

Go to page: First ... 39 40 41 42 [43] 44 45 46 47 ... Last