FAQ

  This is a list of the questions which have come up on the newsgroup with
  any answers that were given. (Somewhat edited by yours truly.)  In a few
  cases I have left in the names of the participants.  If you'd like me to
  remove your name, let me know.  If you have other comments/corrects, just
  drop me a line.  (Of course I'll expect *you* to supply any corrections! :-)


-Bil


     ==================================================================
   	   F R E Q U E N T L Y    A S K E D    Q U E S T I O N S
     ==================================================================
				Also see:

     Brian's FAQ
     (Sun's Threads page and FAQ is no more.)

  Many of the most general questions can be answered by reading (a) the
  welcome message, (b) the general information on the other threads pages,
  and (c) any of the books on threads.  References to all of these can be
  found in the welcome message.

Q1:   How fast can context switching be?
Q2:   What about special purpose processors?
Q3:   What kinds of issues am I faced with in async cancellation?
Q4:   When should I use these new thread-safe "_r" functions?
Q5:   What benchmarks are there on POSIX threads?
Q6:   Has anyone used the Sparc atomic swap instruction?
Q7:   Are there MT-safe interfaces to DBMS libraries?
Q8:   Why do we need re-entrant system calls?
Q9:   Any "code-coverage" tools for MT applications?
Q10:  How can POSIX join on any thread?
Q11:  What is the UI equivalent for PTHREAD_MUTEX_INITALIZER?
Q12:  How many threads are too many in one heavyweight process? 
Q13:  Is there an atomic mutex_unlock_and_wait_for_event()?
Q14:  Is there an archive of this newsgroup somewhere?
Q15:  Can I copy pthread_mutex_t structures, etc.?
Q16:  After 1800 calls to thr_create() the system freezes. ??
Q17:  Compiling libraries which might be used in threaded or unthreaded apps?
Q18:  What's the difference of signal handling for process and thread? 
Q19:  What about creating large numbers of threads?
Q20:  What about using sigwaitinfo()?
Q21:  How can I have an MT process communicate with many UP processes?
Q22:  Writing Multithreaded code with Sybase CTlib ver 10.x?
Q23:  Can we avoid preemption during spin locks?
Q24:  What about using spin locks instead of adaptive spin locks?
Q25:  Will thr_create(...,THR_NEW_LWP) fail if the new LWP cannot be added?
Q26:  Is the LWP released upon bound thread termination?
Q27:  What's the difference between pthread FIFO the solaris threads scheduling?
Q28:  I really think I need time-sliced RR.
Q29:  How important is it to call mutex_destroy() and cond_destroy()?
Q30:  EAGAIN/ENOMEM etc. apparently aren't in ?!
Q31:  What can I do about TSD being so slow?
Q32:  What happened to the pragma 'unshared' in Sun C?
Q33:  Can I profile an MT-program with the debugger?
Q34:  Sometimes the specified sleep time is SMALLER than what I want.
Q35:  Any debugger that single step a thread while the others are running?
Q36:  Any DOS threads libraries?
Q37:  Any Pthreads for Linux?
Q38:  Any really basic C code example(s) and get us newbies started?
Q39:  Please put some Ada references in the FAQ.
Q40:  Which signals are synchronous, and whicn are are asynchronous?
Q41:  If we compile -D_REENTRANT, but without -lthread, will we have problems?
Q42:  Can Borland C++ for OS/2 give up a TimeSlice?
Q43:  Are there any VALID uses of suspension?
Q44:  What's the status of pthreads on SGI machines?
Q45:  Does the Gnu debugger support threads?
Q46:  What is gang scheduling?
Q47:  LinuxThreads linked with X11, calls to X11 seg fault.
Q48:  Are there Pthreads on NT?
Q49:  What about garbage collection?
Q50:  Does anyone have any information on thread programming for VMS?
Q51:  Any information on the DCE threads library?
Q52:  Can I implement pthread_cleanup_push without a macro?
Q53:  What switches should be passed to particular compilers?
Q54:  How do I find Sun's bug database?
Q55:  How do the various vendors' threads libraries compare?
Q56:  Why don't I need to declare shared variables VOLATILE?
Q57:  Do pthread_cleanup_push/pop HAVE to be macros (thus lexically scoped)?
Q58:  Analyzer Fatal Error[0]:  Slave communication failure ??
Q59:  What is the status of Linux threads?
Q60:  The Sunsoft debugger won't recognize my PThreads program!
Q61:  How are blocking syscall handled in a two-level system?
Q62:  Can one thread read from a socket while another thread writes to it?
Q63:  What's a good way of writing threaded C++ classes?
Q64:  Can thread stacks be built in privately mapped memory?
Q66:  I think I need a FIFO mutex for my program...
Q67:  Why my multi-threaded X11 app with LinuxThreads crashes?
Q68:  How would we put a C++ object into a thread?
Q69:  How different are DEC threads and Pthreads?
Q70:  How can I manipulate POSIX thread IDs?
Q71:  I'd like a "write" that allowed a timeout value...
Q72:  I couldn't get threads to work with glibc-2.0.
Q73:  Can I do dead-owner-process recovery with POSIX mutexes?
Q74:  Will IRIX distribute threads immediately to CPUs?
Q75:  IRIX pthreads won't use both CPUs?
Q76:  Are there thread mutexes, LWP mutexes *and* kernel mutexes?
Q77:  Does anyone know of a MT-safe alternative to setjmp and longjmp?
Q78:  How do I get more information inside a signal handler?
Q79:  Is there a test suite for Pthreads? 
Q80:  Flushing the Store Buffer vs. Compare and Swap
Q81:  How many threads CAN a POSIX process have? 
Q82:  Can Pthreads wait for combinations of conditions?
Q83:  Shouldn't pthread_mutex_trylock() work in shared memory even if it's NOT PTHREAD_PROCESS_SHARED?
Q84:  What about having a NULL thread ID?
Q85:  Explain Traps under Solaris
Q86:  Is there anything similar to posix conditions variables in Win32 API ?
Q87:  What if a cond_timedwait() times out AND the condition is TRUE?
Q88:  How can I recover from a dying thread?
Q89:  How to implement POSIX Condition variables in Win32?
Q90:  Linux pthreads and X11
Q91:  One thread runs too much, then the next thread runs too much!
Q92:  How do priority levels work?
Q93:  C++ member function as the startup routine for pthread_create(). 
Q94:  Suprious wakeups, absolute time, and pthread_cond_timedwait()
Q95:  Conformance with POSIX 1003.1c vs. POSIX 1003.4a?
Q96:  Cleaning up when kill signal is sent to the thread.?
Q97:  C++ new/delete replacement that is thread safe and fast?
Q98:  beginthread() vs. endthread() vs. CreateThread? (Win32)
Q99:  Using pthread_yield()?
Q100: Why does pthread_cond_wait() reacquire the mutex prior to being cancelled?
Q101: HP-UX 10.30 and threads?
Q102: Signals and threads are not suited to work together?
Q102: Patches in IRIX 6.2 for pthreads support?
Q104: Windows NT Fibers?
Q105: LWP migrating from one CPU to another in Solaris 2.5.1?
Q106: What conditions would cause that thread to disappear?
Q107: What parts, if any, of the STL are thread-safe?
Q108: Do pthreads libraries support cooperative threads?
Q109: Can I avoid mutexes by using globals?
Q110: Aborting an MT Sybase SQL?
Q111: Other MT tools?
Q112: That's not a book. That's a pamphlet!
Q114: How to cleanup TSD in Win32?
Q115: Onyx1 architecture has one problem
Q116: LinuxThreads linked with X11 seg faults.
Q117: Comments about Linux and Threads and X11
Q118: Memory barriers for synchonization
Q119: Recursive mutex debate
Q120: Calling fork() from a thread
Q121: Behavior of [pthread_yield()] sched_yield()
Q122: Behavior of pthread_setspecific()
Q123: Linking under OSF1 3.2: flags and library order
Q124: What is the TID during initialization? 
Q125: TSD destructors run at exit time... and if it crashes?
Q126: Cancellation and condition variables
Q127: RedHat 4.2 and LinuxThreads?
Q128: How do I measure thread timings? 
Q129: Contrasting Win32 and POSIX thread designs
Q130: What does POSIX say about putting stubs in libc?
Q131: MT GC Issues
Q132: Some details on using CMA threads on Digital UNIX 
Q133: When do you need to know which CPU a thread is on?
Q134: Is any difference between default and static mutex initialization? 
Q135: Is there a timer for Multithreaded Programs? 
Q136: Roll-your-own Semaphores 
Q137: Solaris sockets don't like POSIX_C_SOURCE!
Q138: The Thread ID changes for my thread! 
Q139: Does X11 support multithreading ? 
Q140: Solaris 2 bizzare behavior with usleep() and poll() 
Q141: Why is POSIX.1c different w.r.t. errno usage? 
Q142: printf() anywhere AFTER pthread_create() crashes on HPUX 10.x 
Q143: Pthreads and Linux 
Q144: DEC release/patch numbering 
Q145: Pthreads (almost) on AS/400 
Q146: 
Q147: Thread create timings 
Q148: Timing Multithreaded Programs (Solaris) 
Q149: A program which monitors CPU usage? 
Q150: standard library functions: whats safe and whats not? 
Q151: Where are semaphores in POSIX threads? 
Q152: Thread & sproc (on IRIX) 
Q153: C++ Exceptions in Multi-threaded Solaris Process 
Q154: SCHED_FIFO threads without root privileges ? 
Q155: "lock-free synchronization" 
Q156: Changing single bytes without a mutex 
Q157: Mixing threaded/non-threadsafe shared libraries on Digital Unix 
Q158: VOLATILE instead of mutexes? 
Q159: After pthread_cancel() destructors for local object do not get called?!
Q160: No pthread_exit() in Java.
Q161: Is there anyway I can make my stacks red zone protected?
Q162: Cache Architectures, Word Tearing, and VOLATILE
Q163: Can ps display thread names?
Q164: (Not!) Blocking on select() in user-space pthreads.
Q165: Getting functional tests for UNIX98
Q166: To make gdb work with linuxthreads?
Q167: Using cancellation is *very* difficult to do right...
Q168: Why do pthreads implementations differ in error conditions?
Q169: Mixing threaded/non-threadsafe shared libraries on DU
Q170: sem_wait() and EINTR
Q171: pthreads and sprocs
Q172: Why are Win32 threads so odd?
Q173: What's the point of all the fancy 2-level scheduling??
Q174: Using the 2-level model, efficency considerations, thread-per-X
Q175: Multi-platform threading api
Q176: Condition variables on Win32 
Q177: When stack gets destroyed relative to TSD destructors?
Q178: Thousands of mutexes?
Q179: Threads and C++
Q180: Cheating on mutexes
Q181: Is it possible to share a pthread mutex between two distinct processes?
Q182: How should one implement reader/writer locks on files?
Q183: Are there standard reentrant versions of standard nonreentrant functions?
Q184: Detecting the number of cpus
Q185: Drawing to the Screen in more than one Thread (Win32)
Q186: Digital UNIX 4.0 POSIX contention scope
Q187: Dec pthreads under Windows 95/NT?
Q188: DEC current patch requirements
Q189: Is there a full online version of 1003.1c on the web somewhere?
Q190: Why is there no InterlockedGet?
Q191: Memory barrier for Solaris
Q192: pthread_cond_t vs pthread_mutex_t
Q193: Using DCE threads and java threads together on hpux(10.20)
Q194: My program returns enomem on about the 2nd create.
Q195: Does pthread_create set the thread ID before the new thread executes?
Q196: thr_suspend and thr_continue in pthread
Q197: Are there any opinions on the Netscape Portable Runtime?
Q198: Multithreaded Perl
Q199: What if a process terminates before mutex_destroy()?
Q200: If a thread performs an illegal instruction and gets killed by the system...
Q201: How to propagate an exception to the parent thread?
Q202: Discussion: "Synchronously stopping things" / Cheating on Mutexes
Q203: Discussion: Thread creation/switch times on Linux and NT.
Q204: Are there any problems with multiple threads writing to stdout?
Q205: How can I handle out-of-band communication to a remote client?
Q206: I need a timed mutex for POSIX
Q207: Does pthreads has an API for configuring the number of LWPs?
Q208: Why does Pthreads use void** rather than void*?
Q209: Should I use poll() or select()?
Q210: Where is the threads standard of POSIX ????
Q211: Is Solaris' unbound thread model braindamaged?
Q212: Releasing a mutex locked (owned) by another thread.
Q213: Any advice on using gethostbyname_r() in a portable manner?
Q214: Passing file descriptors when exec'ing a program.
Q215: Thread ID of thread getting stack overflow? 
Q216: Why aren't my (p)threads preemted?
Q217: Can I compile some modules with and others without _POSIX_C_SOURCE?
Q218: timed wait on Solaris 2.6?
Q219: Signal delivery to Java via native interface
Q220: Concerning timedwait() and realtime behavior.
Q221: pthread_attr_getstacksize on Solaris 2.6
Q222: LinuxThreads: Problem running out of TIDs on pthread_create
Q223: Mutexes and the memory model
Q224: Poor performance of AIO in Solaris 2.5?
Q225: Strategies for testing multithreaded code?
Q226: Threads in multiplatform NT 
Q227: Guarantee on condition variable predicate/pthreads?
Q228: Pthread API on NT? 
Q229: Sockets & Java2 Threads
Q230: Emulating process shared threads 
Q231: TLS in Win32 using MT run-time in dynamically loaded DLLs?
Q232: Multithreaded quicksort
Q233: When to unlock for using thread_cond_signal()?
Q234: Multi-Read One-Write Locking problem on NT
Q235: Thread-safe version of flex scanner 
Q236: POSIX standards, names, etc
Q237: Passing ownership of a mutex?
Q238: NT fibers
Q239: Linux (v.2.0.29 ? Caldera Base)/Threads/KDE 
Q240: How to implement user space cooperative multithreading?
Q241: Tools for Java Programming 
Q242: Solaris 2.6, phtread_cond_timedwait() wakes up early
Q243: AIX4.3 and PTHREAD problem
Q244: Readers-Writers Lock source for pthreads
Q245: Signal handlers in threads 
Q246: Can a non-volatile C++ object be safely shared amongst POSIX threads?
Q247: Single UNIX Specification V2
Q248: Semantics of cancelled I/O (cf: Java)
Q249: Advice on using multithreading in C++?
Q250: Semaphores on Solaris 7 with GCC 2.8.1 
Q251: Draft-4 condition variables (HELP) 
Q252: gdb + linuxthreads + kernel 2.2.x = fixed :) 
Q253: Real-time input thread question
Q254: How does Solaris implement nice()?  
Q255: Re: destructors and pthread cancelation...  
Q256: A slight inaccuracy WRT OS/2 in Threads Primer 
Q257: Searching for an idea 
Q258: Benchmark timings from "Multithreaded Programming with Pthreads" 
Q259: Standard designs for a multithreaded applications? 
Q260: Threads and sockets: Stopping asynchroniously 
Q261: Casting integers to pointers, etc. 
Q262: Thread models, scalability and performance  
Q263: Write threaded programs while studying Japanese!  
Q264: Catching SIGTERM - Linux v Solaris 
Q265: pthread_kill() used to direct async signals to thread? 
Q266: Don't create a thread per client 
Q267: More thoughts on RWlocks 
Q268: Is there a way to 'store' a reference to a Java thread? 
Q269: Java's pthread_exit() equivalent?  
Q270: What is a "Thread Pool"?
Q271: Where did "Thread" come from?
Q272: Now do I create threads in a Solaris driver?
Q273: Synchronous signal behavior inconsistant?
Q274: Making FORTRAN libraries thread-safe?
Q275: What is the wakeup order for sleeping threads?
Q276: Upcalls in VMS?
Q277: How to design synchronization variables?
Q278: Thread local storage in DLL?
Q279:  How can I tell what version of linux threads I've got?
Q280: C++ exceptions in a POSIX multithreaded application?
Q281: Problems with Solaris pthread_cond_timedwait()?
Q282: Benefits of threading on uni-process
Q283: What if two threads attempt to join the same thread?
Q284: Questions with regards to Linux OS?
Q285: I need to create about 5000 threads?
Q286:  Can I catch an exception thrown by a sla
Q287: _beginthread() versus CreateThread()?


================================================================
 Q1: How fast can context switching be?  

In general purpose processors (SPARC, MIPS, ALPHA, HP-PA, POWER, x86) a
LOCAL thread context switch takes on the order of 50us.  A GLOBAL thread
context switch takes on the order of 100us.  However...

heddaya@csb.bu.edu (Abdelsalam Heddaya) writes:

>- Certain multi-threaded processor architectures, with special support
>  for on-chip caching of thread contexts can switch contexts in,
>  typically, less than 10 cycles, down to as little as one cycle.

The Tera machine switches with 0 cycles of overhead.

>  Such processors still have to incur a high cost when they run out of
>  hardware contexts and need to perform a full "context swap" with
>  memory.

Hmmm.  With 128 contexts/processors and 16 processors on the smallest
machine, we may be talking about a rare sitution.  Many people doubt
we'll be able to keep the machine busy, but you propose an
embarassment of riches/parallelism.

In any case, I disagree with the implication that a full context swap
is a problem to worry about.  We keep up to 2048 threads active at a
time, with others confined to memory.  The processors issues
instructions for the active threads and completely ignore the inactive
threads -- there's no swapping of threads between processor and memory
in the normal course of execution.  Instead, contexts are "swapped"
when one thread finishes, or blocks too long, or is swapped to disk,
etc.  In other words, at fairly significant intervals.

Preston Briggs

================================================================

 Q2: What about special purpose processors?  

What are the distinctions between these special purpose processors and
the general purpose processors we're using?


??

================================================================
 Q3: What kinds of issues am I faced with in async cancellation?  


Michael C. Cambria wrote:
> 
> In article <4eoe2a$kje@news.hal.com>, spike@hal.com (Spike White) wrote:
> [deleted]
> > thread2()
> > {
> >    ...
> >    while(1) {
> >       pthread_setasynccancel(CANCEL_ON);
> >       pthread_testcancel();  /* if there's a pending cancel */
> >       read(...);
> >       pthread_setasynccancel(CANCEL_OFF);
> >       ...process data...
> >    }
> > }
> >
> > Obviously, you shouldn't use any results from the read() call that was
> > cancelled -- God knows what state it was when it left.
> >
> > That's the only main use I've ever found for async cancel.
> 
> I used something quite similar to your example (quoted above) in my
> original question.
> 
> Since the read() call itself is not async cancel safe according to Posix,
> is it even safe to do the above?  In general for any posix call which is
> not async cancel safe, my guess (and many e-mails to me agree) is to
> just not use it.
> 
> Using read() as an example, I'll bet everyone will agree with you not
> to use the results of the read() call.  However, the the motivation for
> my original question was, being as a call() is not async cancel safe,
> by canceling a thread when it is in one of these calls _may_ screw up
> other threads in general and other threads using the same fd in
> particular.  This is why I asked why one would use it.
> 
> In your example, if read() did anything with static data, the next read on
> that fd could have problems if a thread was cancelled while in the read().
> (Note:  if you don't like the "static data" example, substitute whatever
> you like for the implementation reason for read(), or any call, not being
> async cancel safe.  I used static data as an example only.)
> 
> Mike

Specifically, NO, it is NOT safe to call read() with async cancel. On some
implementations it may work, sometimes. In general, it *MAY* work if, on
the particular release of your particular operation system, read() happens
to be implemented with no user-mode code (aside from a syscall trap). In
most cases, a user mode cancel will NOT be allowed to corrupt kernel data.

However, no implementations make any guarantees about their implementation
of read(). It may be a syscall in one version and be moved partly into
libc in the next version.

Unfortunately, the OSF DCE porting guide made reference to the possibility
of using async cancel in place of synchronous system cancel capability on
platforms that don't support the latter. That was really too bad, and it
set a very dangerous precedent.

POSIX 1003.1c-1996 encourages all routines to document whether they are
async cancel safe. (Luckily the advice is in rationale -- which is to say
it's really just commentary and not part of the standard -- because it'd
be horrendously difficult to change the documentation of every single
routine in a UNIX system.) In practice, you should always assume that a
function is NOT async cancel safe unless it says that it IS. And you won't
see that very often.

Because, as has already been commented, async cancel really isn't very
useful. There is a certain small class of application that can benefit
dramatically from async cancel, for good response to shutdown requests in
long-running compute-bound threads. In a long and tight loop it's not
practical to call pthread_testcancel(). So in cma we provided async cancel
for those cases. In retrospect I believe that's probably one of the bad
parts of cma, which POSIX should have omitted. There may well have been
"hard realtime" people in the room who wanted to use it, though (the POSIX
threads standard was developed by roughly 10 "threads people" and 40 to 50
"realtime people").

------------------------------------------------------------------------
Dave Butenhof                              Digital Equipment Corporation
butenhof@zko.dec.com                       110 Spit Brook Rd, ZKO2-3/Q18
Phone: 603.881.2218, FAX: 603.881.0120     Nashua, NH 03062-2711
                 "Better Living Through Concurrency"
------------------------------------------------------------------------


> In article <3659533E.ABBA639A@hotmail.com>,
> Jose Luis Ramos =?iso-8859-1?Q?Mor=E1n?=   wrote:
> %   pthread_setcancelstate(PTHREAD_CANCEL_ENABLE,NULL);
> %   pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS,NULL);
>
> I would guess that your problem comes from this. Asynchronous cancellation
> is almost never a good idea, but if you do use it, you should be really
> careful about whether there's anything with possible side-effects in your
> code. For instance, the C++ exception handler could be screwed up for your
> whole process if you cancel at a bad moment.
>
> Anyway, try taking out the asynchronous cancellation and see if the problem
> goes with it.

I'll put it a little more strongly than Patrick. The program is illegal. You
CANNOT call any function with asyncronous cancel enabled unless that function
is explicitly specified as "async-cancel safe". There are very few such
functions, and sleep() is not one of them. In fact, within the scope of the
POSIX and UNIX98 standards, with async cancel enabled you are allowed only to

  1. Disable asynchronous cancellation (set cancel type to DEFERRED)
  2. Disable cancellation entirely (set cancel state to DISABLE)
  3. Call pthread_cancel() [This is bizarre and pointless, but it is specified
     in the standard.]

If you call any other function defined by ANSI C, POSIX, or UNIX98 with async
cancel enabled, then your program is nonportable and "non conforming". It MAY
still be "correct", but only IF you are targeting your code to one specific
implementation of the standard that makes the NON-portable and NON-standard
guarantee, in writing, that the function you're calling actually is
async-cancel safe on that implementation. Otherwise, the program is simply
broken.

You can, of course, write your own async-cancel safe functions. It's not that
hard to do. In general, like most correct implementations of pthread_cancel(),
you simply DISABLE async cancellation on entry and restore the previous
setting on exit. But it's silly to do that very often. And, of course, that's
not the same as actually allowing async cancel. THAT is a much, much harder
job, except for regions of code that own no resources of any kind.

Asynchronous cancelation was designed for tight CPU-bound loops that make no
calls, and therefore would suffer from the need to call pthread_testcancel()
on some regular basis in order to allow responsiveness to cancellation
requests. That's the ONLY time or place you should EVER even consider using
asynchronous cancellation.

/---------------------------[ Dave Butenhof ]--------------------------\
| Compaq Computer Corporation                     butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/


================================================================
 Q4: When should I use these new thread-safe "_r" functions?  


David Brownell wrote:
> 
> If the "_r" versions are available at all times, use them but
> beware of portability issues.  POSIX specifies a pretty minimal
> set and many implementations add more (e.g. gethostbyname_r).
> Some implementations only expose the "_r" versions if you
> compile in a threaded environment, too.
> 
> - Dave

POSIX 1003.1c-1995 deliberately separates _POSIX_THREAD_SAFE_FUNCTIONS
from _POSIX_THREADS so that they can be easily implemented by
non-threaded systems. The "_r" versions aren't just thread-safe, they
are also much "cleaner" and more modular than the traditional forms.
(for example, you can have several independent readdir_r or strtok_r
streams active simultaneously).

The grand vision is that all UNIX systems, even though without threads,
would of course want to pick up this wonderful new set of interfaces. I
doubt you'll see them in any system without threads support, of course,
but it would be nice.

================================================================
 Q5: What benchmarks are there on POSIX threads?  

In the book on POSIX.4 by B.Gallmeister there are some very useful POSIX
benchmark programs which allow to measure the real-time performance of an
operating system. However there is nothing on the threads of POSIX.4a!  Does
anybody know of a useful set of benchmark programs on these POSIX threads ??

Any help is greatly appreciated.

Markus Joos
CERN ECP/ESS
(markus.joos@cern.ch)


??
================================================================
 Q6: Has anyone used the Sparc atomic swap instruction?  

Has anyone used the Sparc atomic swap instruction to safely build lists 
in a multithreaded application?  Any examples?  Any references?


Yes, but it would not help you if you use sun4c machines. ( No atomic
instructions..)  Thus you would be forced to use atomic in sun4m or later,
and spl stuff in sun4c.  Does not make a pretty picture. Why not use
mutex_lock/unlock and let the libraries worry about that. mutex_lock uses
atomic/spl stuff.

Sinan

[sun4c are SPARC v7 machines such as 4/110, SS1, SS1+, SS2, IPC,
IPX, EPC, EPX. sun4m are v8 machines including SS10, SS20, SS4, SS5, 4/690,
SS1000, SC2000. The UltraSPARC machines are SPARC v8+ (soon to be v9), but
have the same instructions as the sun4ms.]
================================================================
 Q7: Are there MT-safe interfaces to DBMS libraries?  

A: In general, no.  My current understanding is that NO major DBMS
   vendor has an MT-safe client-side library. (1996)


Peter Sylvester wrote:
> 
> In article <311855DF.41C6@steffl.aui.dec.com>, Andreas Reichenberger
>  wrote:
> 
> > Richard Moulding wrote:
> > >
> > > I need to interface to an Oracle 7 DB from DCE (non-Oracle)
> > > clients. We are planning to build our own  DCE RPC-stored
> > > procedure interface but someone must be selling something to do
> > > this, no?
> 
> I have the same problem with an Informix application which uses a DCE
> interface, and currently limit it to 1 thread coming in.  This works, but
> could be a bottleneck in busy environments, as other incoming RPCs are put
> in a queue (blocked) until the current one finishes.

... stuff deleted

 
> A potential way around this would be to fork off separate processes which
> then start their own connection to the database.  The parent then acts as a
> dispatcher for requests coming in.  I know the forking part works without
> DCE, but I suspect that you have to do all the forks before going into the
> DCE server listening mode.
> 
> I also thought I heard something about Oracle providing a thread safe
> library, maybe in 7.3.  Anyone know?
> 
> --
> Peter Sylvester
> MITRE Corp.
> Bedford, MA
> (peters@mitre.org)

This is exactly the way we handled the problem. We wrote a tool that
generates the complete dispatcher from an IDL file. The dispatcher (which is
virtually invisible to the clients and to the developers) distributes the
requests from the clients to its 'backends', which are connected to the
DB. The backends are implemented as single-threaded DCE Servers with the
Interface specified in the IDL File.

We added some features that are not in DCE, like 
  - asyncronous RPC's (the RPC returns immediately and the client can ask the
    dispatcher to return the state of the RPC (if it is done or still running)
    or request the RPC to be canceled) 
  - dividing the backends into classes. i.e. it's possible to have one class of
    backends for querying the database and another class for updates, etc. By
    assigning 2 backends to the query class and the rest of the backends to
    other classes you can limit the number of concurrent queries to 2 (because
    they are time consuming). The client has to specify which class is to be used
    for a RPC (we currently support up to 10 classes)

Context handles are used to tie a client to one backend for transactions which
require more than one RPC to be handled by the same backend (= DB Session).

The reason why the hell we had to do this anyway was to limit the number of
backend processes neccessary to support a few hundred PC clients. We
currently run it on AIX and Digital UNIX with Oracle and Ingres. However,
there's no reason why it shouldn't work on any UNIX platform which supports
OSF DCE (V1.1) and with any DB.

Feel free to contact me for more details...

See 'ya

================================================================
 Q8: Why do we need re-entrant system calls?  

A:
jbradfor@merlot.ecn.purdue.edu (Jeffrey P Bradford) wrote:
>Why do we need re-entrant system calls?  I know that it's so that
>system calls can be used in a multithreaded environment, but how often
>does one really have multiple threads executing the same system call?
>Do we really need system calls that can be executed by multiple
>threads, or would mutual exclusion be good enough?

Well, there have been some implimentations that felt (feel?) that mutual
exclusion is good enough. And, in fact, that will "thread safe" the
functions. But it runs havoc with performance, and things like
cancelability. Turns out that real applications have multiple threads calls
executing the same system call all the time. read() and write() are popular,
as are send() and recv() (On UNIX).

>I'm assuming that system calls can be designed intelligently enough so
>that, for example, if a process wants to perform a disk read, the
>process performs a system call, exits the system call (so another
>thread can perform a disk read), and then is woken up when the disk
>read is done.
>
>Jeff

[I assume the behavior you reference "leave the system call" means
"return to user space"]

That all depends on the OS. On UNIX, that is not the default
system call behavior. On VMS it is (Just two examples).

Brian Silver.

================================================================
 Q9: Any "code-coverage" tools for MT applications?  

Is there an application that can help me with "code-coverage" for
MT applications?


A:

Upon which platform are you working?  I did performance profiling last week
on a MT app using prof & gprof on a Solaris 2.4 machine.  For code coverage,
I use tcov.  I suspect that most OS's w/ kernel threads have thread-aware
gprof and tcov commands.

--
Spike White          | spike@hal.com               | Biker Nerds
HaL Software Systems | '87 BMW K75S, DoD #1347     |  From  HaL
Austin, TX           |  http://www.halsoft.com/users/spike/index.html 
Disclaimer:  HaL, want me to speak for you?  No, Dave... 
================================================================
 Q10: How can POSIX join on any thread?  

The pthread_join() function will not allow you to wait for "any" 
thread, like the UI function thr_join() will.  How can I get this?

A:
> >: I want to create a number of threads and then wait for the first
> >: one to finish, not knowing which thread will finish first.  But
> >: it appears pthread_join() forces me to specify exactly which of
> >: my threads I want to wait for.  Am I missing something basic, or
> >: is this a fundamental flaw in pthread_join()?
> >
> >:      Rich Stevens
> >
> >Good call.  I notice Solaris native threads have this support and the
> >pthreads implementations I've seen don't.  I wondered about this myself.
> >
> 
> Same here.  The situation I ran into was a case where once the main
> created the necessary threads and completed any work it was responsible
> for, it just needed to "hang-around" until all the threads completed
> their work before exiting.  pthread_join() for "any" thread in loop using
> a counter for the number of threads seemed the logical choice.  Then I
> realized Solaris threads supported this but POSIX didn't (along with
> reader/writer locks).  Oh well.
> 
> How about the Solaris SPLIT package.  Does it support the "wait for any"
> thread join?

This "wait for any" stuff is highly misleading, and dangerous in most real
threaded applications. It is easy to compare with the traditional UNIX "wait
for any process", but there's no similarity. Processes have a PARENT -- and
when a process "waits for any" it is truly waiting only for its own
children. When your shell waits for your "make" it CANNOT accidentally chomp
down on the termination of the "cc" that make forked off!

This is NOT true with threads, in most of the common industry threading
models (including POSIX 1003.1c-1995 and the "UNIX International" threads
model supported by Solaris). Your thr_join(NULL,...) call may grab the
termination status of a thread used to parallelize an array calculation
within the math library, and thus BREAK the entire application.

Without parent/child relationships, "wait for any" is not only totally
useless, it's outright dangerous. It's like the Win32 "terminate thread"
interface. It may seem "neat" on the surface, but it arbitrarily breaks all
shared data & synchronization invariants in ways that cannot be detected or
repaired, and thus CANNOT be used in anything but a very carefully
constructed "embedded system" type environment where every aspect of the
code is tightly controlled (no third-party libraries, and so forth). The
very limited enviroments where they are safe & useful are dramatically
outweighed by the danger that having them there (and usually very poorly
explained) encourages their use in inappropriate ways.

It really wouldn't have been hard to devise POSIX 1003.1c-1995 with
parent/child relationships. A relatively small overhead. It wasn't even
seriously considered, because it wasn't done in any of the reference
systems, and certainly wasn't common industry practice. Nevertheless,
there are clearly advantages to "family values" in some situations...
among them being the ability to usefully support "wait for any". But
wishful thinking and a dime gets you one dime...

------------------------------------------------------------------------
Dave Butenhof                              Digital Equipment Corporation
butenhof@zko.dec.com                       110 Spit Brook Rd, ZKO2-3/Q18
Phone: 603.881.2218, FAX: 603.881.0120     Nashua, NH 03062-2711
                 "Better Living Through Concurrency"
------------------------------------------------------------------------

I find Dave's comments to be most insightful.  He hits on a big point
that I have hear a number of people express confusion about.  My 2-bits
to add:

  As a programmer we should be thinking about the availability of resources
-- when is something ready for use?  "Is the Matrix multiply complete?" "Has
the data request been satisfied?" etc.  thr_join() is often used as a cheap
substitute for those questions, because we ASSUME that when all N threads
have exited, that the computation is complete.  (Generally accurate, as long
as we control the entire program.  Should some lout get hired to maintain
our code, this assumption could become false in a hurry.)

  The only instance where we REALLY care if a thread has exited is when
the resource in question IS that thread (e.g., we want to un-mmap pages
we reserved for the stack or other rare stuff).

  So... the correct answer is "Don't do that."  Don't use thr_join()
to count threads as they exit.  Set up a barrier or a CV and have the threads
count down as they complete their work.  IE:

worker threads:

	do_work();
...  	lock(M);
	running_threads--;
	if (running_threads == 0) cond_signal(CV);
	unlock(M);
	thr_exit();




"Master" thread:

...	running_threads = N;
	create_workers(N);
	lock(M)
	while (running_threads != 0) cond_wait(M, CV);
	...


-Bil


================================================================
 Q11: What is the UI equivalent for PTHREAD_MUTEX_INITALIZER?  

A:

From the man page (man mutex_init):

Solaris Initialize
     The equivalent Solaris API used to  initialize  a  mutex  so
     that  it has several different types of behavior is the type
     argument passed to mutex_init().  No current type  uses  arg
     although  a  future  type  may  specify  additional behavior
     parameters via arg.  type may be one of the following:

     USYNC_THREAD        The mutex can synchronize  threads  only
                         in  this  process.  arg is ignored.  The
                         USYNC_THREAD Solaris mutex type for pro-
                         cess  scope  is  equivalent to the POSIX
                         mutex         attribute          setting
                         PTHREAD_PROCESS_PRIVATE.

     USYNC_PROCESS       The mutex  can  synchronize  threads  in
                         this  process and other processes.  Only
                         one process should initialize the mutex.
                         arg   is   ignored.   The  USYNC_PROCESS
                         Solaris mutex type for process scope  is
                         equivalent  to the POSIX mutex attribute
                         setting   PTHREAD_PROCESS_SHARED.    The
                         object  initialized  with this attribute
                         must  be  allocated  in  memory   shared
                         between  processes, i.e. either in Sys V
                         shared memory  (see  shmop(2)).   or  in
                         memory  mapped  to a file (see mmap(2)).
                         It is illegal to initialize  the  object
                         this  way and to not allocate it in such
                         shared memory.

     Initializing mutexes can also be accomplished by  allocating
     in  zeroed  memory  (default),  in  which  case,  a  type of
     USYNC_THREAD is assumed.  The same mutex must not be  simul-
     taneously  initialized  by  multiple  threads.  A mutex lock
     must not be re-initialized while in use by other threads.

     If default mutex attributes are used, the macro DEFAULTMUTEX
     can  be used to initialize mutexes that are statically allo-
     cated.

================================================================
 Q12: How many threads are too many in one heavyweight process?    

How many are too many for a single machine?

A:

The answer, of course, is "it depends".

Presumably, the number of threads you're considering far outstrips the
number of processors you have available, so it's not really important
whether you're running on uni- or a multiprocessor, and it's not really
important (in this general case) whether the threads implementation has
any kernel support (presumably it doesn't on HP-UX, judging by your post
from 14 Feb 1996 14:31:42 -0500).  So, it comes down to what these
bazillion threads of yours are actually doing.  

If, for the most part, they just sit there waiting for someone to tickle
the other end of a socket connection, then you can probably create LOTS
before you hit "too many".  In this case it would depend on how much
memory is available to your process, in which to keep all of these
sleeping threads (and how much kernel resources are available to create
sockets for them ;-).

If, on the other hand, every one of these bazillion threads is hammering
away on the processor (trying to compute some fractal or something :-),
then creating any more threads than you have processors is too many.
That is, you waste time (performance, throughput, etc.) in switching
back and forth between the threads which you could be spending on
something useful.  That is, life would be better if you just created a
couple of threads and had them make their way through all the work at
hand.

Presumably, your application falls somewhere between the two extremes.
The idea is to design so that your "typical operating conditions"
involve a relatively small number of threads active at any one time.
Having extra ones running isn't a catastrophe, it just means that things
aren't quite as efficient as they otherwise might be.

-- 

------------------------------------------------------------------------
Webb Scales                                Digital Equipment Corporation
scales@wtfn.enet.dec.com                   110 Spit Brook Rd, ZKO2-3/Q18
Voice: 603.881.2196, FAX: 603.881.0120     Nashua, NH 03062-2711
         Rule #12:  Be joyful -- seek the joy of being alive.
------------------------------------------------------------------------

================================================================
 Q13: Is there an atomic mutex_unlock_and_wait_for_event()?  

Is it possible for a thread to release a mutex and begin
waiting on an "event" in one atomic operation?  I can think of a few
convoluted ways to achieve or simulate this, but am wondering if
there's an easy solution that I'm missing.


A:

This isn't how you'd really want to look at things (POSIX). Figure out what
condition you're interested in and use a CV.

	================================================

The NT4.0 beta has a new Win32 API, SignalObjectAndWait that will do what you
want. Sorry, it is not available in 3.51 or earlier.
    -John

Robert V. Head
================================================================
 Q14: Is there an archive of this newsgroup somewhere?  

I believe http://www.dejanews.com keeps a 1 year record of every
newsgroup on the Usenet.  You can search it by author to get your
articles, then pick out individual threads...

================================================================
 Q15: Can I copy pthread_mutex_t structures, etc.?  

"Ian" == Ian Emmons  writes:
In article <32D4149F.588B@persistence.com> Ian Emmons  writes:

Ian> Variables of the data type pthread_t are, semantically speaking, a sort of 
Ian> reference, in the following sense:

Ian> 	pthread_t tid1;
Ian> 	pthread_t tid2;
Ian> 	void* ret_val;

Ian> 	pthread_create(&tid1, NULL, some_function, NULL);
Ian> 	// Now tid1 references a new thread.
Ian> 	tid2 = tid1;
Ian> 	// Now tid2 references the same thread.
Ian> 	pthread_join(tid2, &ret_val);

Ian> In other words, after creating the thread, I can assign from one pthread_t 
Ian> to another, and they all reference the same thread.  Pthread_key_t's (I 
Ian> believe) behave the same way.

	You should not copy one structure pthread_t to another pthread_t
...  it may not be portable.  In some implementations the pthread_t is
not simple a structure containing only a pointer and some keys .... it
is infact the REAL structure, which would then create two independant
structures which each can be manipulated individually reaping havoc.

Ian> An attributes object, like pthread_attr_t (or an unnamed semaphore sem_t), 
Ian> on the other hand does not behave this way.  It has value semantics, because 
Ian> you can't copy one into another and expect to have a second valid attribute 
Ian> object.

Ian> My question is, do pthread_mutex_t's and pthread_cond_t's behave as 
Ian> references or values?

	Same statement .... I have seen enough problems where someone copied
an initialized lock then continued to lock the two mutexes independently
creating very unwanted behavior.

-- 
William E. Hannon Jr.                         internet:whannon@austin.ibm.com
AIX/DCE Technical Lead                                         whannon@austin
Austin, Texas 78758     Department ATKS/9132     Phone:(512)838-3238 T/L(678)
'Confidence is what you had, before you understood the situation.' Dr. Dobson


FOLLOWUP: For most programs, you should be passing pointers around, not
structures:


pthread_mutex_t		my_lock;


main()
{  ...
   foo(&my_lock);
   ...
}

foo(pthread_mutex_t *m)
{
pthread_mutex_lock(m);
...
}
================================================================
 Q16: After 1800 calls to thr_create() the system freezes. ??  

My problem is that the thread does not get freed or released back to the
system for reuse.  After 1800 calls to thr_create() the system freezes. ??
A: The default for threads in both UI and POSIX is for threads to be
   "undetached" -- meaning that they MUST be joined (thr_join()).  Otherwise
   they will not be garbage collected.  (This default is the wrong choice.  Oh
   well.)
================================================================
 Q17: Compiling libraries which might be used in threaded or unthreaded apps?  


   What *is* the straight scoop on how to compile libraries which 
   might be used in threaded or unthreaded apps?  Hopefully the 
   "errno" and "putc()" macros will continue to work even if
   libthread isn't pulled in, so that vendors can make a single
   version of any particular library.

A: Always compile *all* libraries with the reentrancy flag (_REENTRANT for
   UI threads, _POSIX_C_SOURCE=199506L for POSIX threads). Otherwise some 
   poor soul will try to use your library and get hammered.  putc() and
   getc() WILL be slower, but you may use putc_unlocked() & getc_unlocked()
   if you know the I/O stream will be used safely.

   All Solaris libraries are compiled like this.
================================================================
 Q18: What's the difference of signal handling for process and thread?   

   What's the difference of signal handling for process and thread? Do the
   signals divided into the types of process-based and thread-based which were
   treated differently in HP-RT? Is there any examples? I'd like to know how to
   initiate, mask, block, wait, catch, ...... the signals. How can I set the
   notification list (process or thread?) of SIGIO for both socket and tty
   using fcntl or ioctl? 

A: You probably want to buy one of the books that discuss this in detail.
   Here's the short answer:



	Signal masking is on per-thread based.
	But the signal handlers are per-process based.
	The synchronous signals like SIGSEGV, SIGILL etc will be 
	processed by the thread which caused the signal.

	The other signals will be handled by any ready thread which
	has the mask enabled for the signal.
	
	There are no special thread library for signal handling.
================================================================
 Q19: What about creating large numbers of threads?  

I've asked a question about creating 2500 unbound threads. During these
days, I have written some more testing programs. Hope you would help me to
solve some more problems.

1. I have written a program that creates 10 threads. Then the 10 threads
each create 10 more threads. The 100 newly created threads each creates 10
more threads. In a SPARC 2000, if the concurrency level is 100, the program
takes 7 seconds to terminate. From a paper, unbound thread creation is
claimed to take only 56 usec. How comes my testing program is so slow on a
SPARC 2000 that has 20 CPUs? If I use a SPARC 10, the program only takes 1
second to terminate. Is SPARC 2000 slower than a SPARC 10?

2. Instead of creating 2500 threads, I have written a program that creates
200 threads and then kills them all and creates 200 threads and kills them
all and ..... After some while of creating and killing, the program hangs. I
use sigaction to set a global signal handler for the whole process. As the
program is so simple, I don't know where the problem is.

3. In addition, I have written a program that creates 1000 bound
threads. Each thread has a simple loop:

		while (1)
		{
			randomly read an entry of an array
		}

   This time, not only my program hangs, the whole SPARC 2000 hangs. I can't
reset the machine from console. Finally, I have to power down the machine.

Thanks in advance.


A:
================================================================
 Q20: What about using sigwaitinfo()?  

>Here is what I am doing.  I am using the early access POSIX threads.
>My main program blocks SIGUSR1 and creates a number of threads.
>One of these threads is dedicated to this signal.  All it does is a
>sigwaitinfo on this signal, sets a flag when it returns, and exits.
>If I send the SIGUSR1 signal to the process using the kill command
>from another window, it does not seem to get it and the other threads
>(which are doing a calculation in a loop) report that SIGUSR1 is not
>pending.
>
>An earlier version of the program which used a signal handler to set
>the flag worked perfectly.
>
>Do you have any ideas on this?

A:

I assume you are using sigwaitinfo(3r) from libposix4.
Unfortunately, sigwaitinfo() is not MT-safe, i.e. does not work correctly
in an MT program, on 2.3/2.4. Use sigwait(2) - it should work on 2.3/2.4.
On 2.5 beta, sigwaitinfo() works.

If you really need the siginfo on 2.3/2.4, it is going to be hard, and the 
solution depends on whether you are running 2.3/2.4 but here is an 
alternative suggestion:

Programmers have used signals between processes as an IPC mechanism. Sounds
like you are trying to do the same. If this is the case, I would strongly
suggest that you use shared memory (see mmap(2)) between processes and
shared memory synchronization (using the SysV shared semaphores - see
semop(2)), or POSIX synchronization objects with the PTHREAD_PROCESS_SHARED
attribute. For example, you can set-up a region of shared memory protected
by a mutex and condition variable. The mutex and condition variable would
also be allocated from the shared memory and would be initialized with the
PTHREAD_PROCESS_SHARED attribute. Now, processes which share this memory
can use the mutex and condition variable as IPC mechanisms - any information
that needs to be passed between them can be passed through the shared
memory (alternative to siginfo :-)). To make this asynchronous, you can
have a thread dedicated to monitoring the shared memory area by waiting
on the condition variable. Now, whenever the signalling process wants to
send a signal, it instead issues a cond_signal on the condition variable.
The thread sleeping on this in the other (receiving) process wakes up
now and processes the information.

In general, signal handlers and threads, even though the system might support
this correctly, should not be used together. Signal handlers could be
looked upon as "substitute threads" when threads were not around in UNIX, 
and now that they are, the interactions between them can be complicated. 
You should mix them together only if absolutely necessary.

================================================================
 Q21: How can I have an MT process communicate with many UP processes?  

>I have a multithreaded process, each thread in the multithreaded
>process wants to communicate with another single-threaded process,
>what is the good way to do that?
>
>Assume each thread in the multithreaded process is identical, i.e.
>they are generated using the same funcation call and each thread
>creates a shared memory to do the communication, will the generated
>shared memories operate independently if no synchronization provided?  

A:


  It sounds like you have the right idea.  For each thread/process pair,
build a shared memory segment and use that for communications.  You'll need
some sort of synchronization variable in that shared segement for
coordination.  

  There is no interaction between segments what-so-ever.
================================================================
 Q22: Writing Multithreaded code with Sybase CTlib ver 10.x?  


>A customer is trying to write a multi-threaded application that also
>uses Sybase CTlib ver 10.x, and he is facing some limitations due to
>the Sybase library. 
>
>BOTTOM LINE: CTlib is reentrant, but according to Sybase is not usable
>in a multi-threaded context. That means it does NOT seem to be usable
>in an MT application.
>
>The purpose of this mail is NOT to get a fix for CTlib, but to try to
>find a workaround, if one exists...

A:

The workaround for the moment is to use the XA library routines from
Sybase, which are, in turn, based upon the TransArc package pthread*
routines.

We should be getting an alpha version of MT safe/hot CTlib towards the first
part of June 1995.  Also of potential interest is there will also be an early
version of native-threaded OpenServer soon as well, which really opens
up a lot of possibilities.

Chris Nicholas
SunSoft Developer Engineering
--------------------------------------------------------------
================================================================
 Q23: Can we avoid preemption during spin locks?  

> 	A while ago I asked you for information on preemption control
> interfaces (in-kernel) which might be available in Solaris2.x. I am
> looking for ways of lowering number of context switches taken as the
> result of adaptive muxtex contention. We have a number of places a
> lock is taken and held for a few scant lines of C. It would be great
> to prevent preemption during these sections of code.

A:

  You're obvious writing a driver of some sort. (Video driver I'd guess?)
And you're VERY concerned with performance on *MP* machines (UPs be damned).
You have tested you code on a standardized, repeatable benchmark, and you
are running into a problem.  You have solid numbers which you are absolutely
certain of.  Right?

  You'll have to excuse my playing the heavy here, but you're talking deep
do-do here, and I don't want to touch it unless I'm totally convinced I (and
you) have to.

  You could set the SPL up to turn off all interrupts.  It would slow your
code down quite a bit though.  The probablity of preemption occuring over "a
few scant lines of C" (i.e., a few dozen instructions) approaches zero.
Regularly suffering from preemption during just these few instructions would
be a VERY odd thing.  I am hard pressed to INVENT a situation like this.
Are you absolutely, totally, completely, 100% certain you're seeing this?
Are you willing to put $10 on it?

================================================================
 Q24: What about using spin locks instead of adaptive spin locks?  
> 
> 	I also would like to know more about something I saw in
> /usr/include/sys/mutex.h. It would appear that it possible to 
> create pure spinning locks (MUXTEX_SPIN) as opposed to the default 
> adaptive mutexes (MUTEX_ADAPTIVE_STAT). These might provide the kind 
> of control I am looking for assuming that these are really supported 
> and not some bastard orphan left over.

A:

  If I understand the question, the answer is "no".  That's what an adaptive
mutex is for.  It optimizes a spin lock to sleep if there's no value in
spinning.  If you use a dumb spin lock instead, you are GUARANTEED to run
slower.
================================================================
 Q25: Will thr_create(...,THR_NEW_LWP) fail if the new LWP cannot be added?  

>	Does Sun's implementation of thr_create(...,THR_NEW_LWP) fail
>to create the multiplexed thread if the new LWP cannot be added to the
>multiplexing pool?  The unixware docs indicate Novell's implementation
>of thr_create() uses THR_NEW_LWP as a hint to the implementation to
>increase the pool size.  They also do not state the behavior if the
>new lwp cannot be created.  What is the official statement?

A:

  It should not create a new thread if it returns EAGAIN.  Mind you, you're
fairly unlikely EVER to see this happen in a real program.  (You'll see it
in bugs & in testing/design.)
================================================================
 Q26: Is the LWP released upon bound thread termination?  

>  In the sun implementation, if you create a bound
>thread, and the thread eventually terminates, is the LWP released
>upon termination, or upon thr_join with the terminated thread?

A:

  Yes, a bound thread's LWP is released.  This should not affect your
programming at all.  Use thr_setconcurrency() & leave it at that.
================================================================
 Q27: What's the difference between pthread FIFO the solaris threads scheduling?  

A:  Very little.

================================================================
 Q28: I really think I need time-sliced RR.  

>Well, i really think I need time-sliced RR. Since I'm making an 
>multithreaded implementation of a functional concurrent process-
>oriented  language. MT support is needed to get good usage
>of multi CPU machines and better realtime. Today processes are custom 
>user-level and the runtime system delivers the scheduling. And the
>language semantic is that processes are timesliced RR.
>Changing the sematic is not realistic. I really hope the pthreads
>will spec RR timeslicing, it would make things easier.

A:

  Think VERY carefully.  When will you ever *REQUIRE* RR scheduling?  And
why?  Remember, you've never had it ever before, so why now?  (There may be
a reason, but it had better be good.)  Scheduling should normally be
invisible, and forcing up to user-awareness is generally a bad thing.

>For the moment, since this will only be a prototype, bound threads
>will do but bot in a real system with a couple with houndreds of
>threads/processes.
>
>Convince me I don't need RR timeslicing, that would make things easier.
>Or how do I make my own scheduler in solaris, or should I stay with
>bound threads?

  OK.  (let me turn it around) Give one example of your program which will
fail should thr 3 run before thr 2 where there is absolutely NO
synchronization involved.  With arbitrary time-slicing of course.  I can't
think of an example myself.  (It's just such a weird depencency that I
can't come up with it.  But I don't know everything...)
================================================================
 Q29: How important is it to call mutex_destroy() and cond_destroy()?  

here is how I init serval of my threading variables

    mutex_init( &lock, USYNC_PROCESS, 0 );
    cond_init( ¬Busy, USYNC_PROCESS, 0 );
   
The storage for the variables is in memory mapped file. once I have
opened the file, I call unlink to make sure it will be automatically
cleaned up. How important is it to call mutex_destroy() and
cond_destroy()? Will I wind up leaking some space in the kernel is I
do not call these functions?

A:
================================================================
 Q30: EAGAIN/ENOMEM etc. apparently aren't in ?!  

A:
  'Course not.  :-)

  They're in errno.h.  pthread_create() will return them if something goes
wrong.  Be careful, ERRNO is NOT used by the threads calls.
================================================================
 Q31: What can I do about TSD being so slow?  
 Q32: What happened to the pragma 'unshared' in Sun C?  

   I read about a pragma 'unshared' for the C-compiler in some Solaris-thread
   papers. The new C-3.01 don't support the feature anymore I think. There is
   no hint in the Solaris 2.4 Multithread Programming Guide. But the new
   TSD is very slow. I tested a program with direct register allocation under
   gcc (asm "%g3") instead of calling the thr_getspecific procedure and it was 
   over three times faster. Can I do the same thing or something else with the 
   Sun C-compiler to make the C-3.01 Code also faster?

A:

The "thread local storage" feature that was mentioned in early papers
about MT on Solaris, and the pragma "unshared", were never
implemented.  I know what you mean about the performance of TSD.  It
isn't very fast.  I think the key here is to try to structure your
program so that you don't rely too much on thread specific data, if
that's possible.

The SPARC specification reserves those %g registers for internal use.
In general, it's dangerous to rely on using them in your code.
However, SC3.0.1 does not use the %g registers in any user code. It
does use them internally, but never across function calls, and never
in user code.  (If you do use the %g registers across function calls,
be sure to save and restore the registers.)

You can accomplish what gcc does with the "asm" statement by writing
what we call an "inline function template."  Take a look at the math
library inline templates for an idea on how to do that, and see the
inline() man page.  You might also want to take a look at the
AnswerBook for SPARC Assembly Language Programming, which is found in
the "Solaris 2.x Software Developer Answerbook".  The latest part
number for that is 

801-6649-10     SPARC ASSEMBLY LANGUAGE REFERENCE MANUAL REV.A AUG 94

The libm templates are found in /opt/SUNWspro/SC3.0.1/lib/libm.il.
Inline templates are somewhat more work to write, as compared to using
gcc's "asm" feature, but, it's safer.  I don't know about the
robustness of code that uses "asm" - I like gcc, and I use it, but
that particular feature can lead to interesting bugs.

Our next compiler, SC4.0 (coming out in late 1995) will use the %g
registers more aggressively, for performance reasons.  (Having more
registers available to the optimizer lets them do more optimizations.)
There will be a documented switch, -noregs=global (or something like
that) that you will use to tell the SC4.0 NOT to use the global
registers.    When you switch to SC4.0, be sure to read the cc(1) man
page and look for that switch.  
================================================================
 Q33: Can I profile an MT-program with the debugger?  

   Can I profile an MT-program with the debugger and a special MT-license
   or do I need the thread-analyser?

A:

The only profiling you can do right now for an MT program is what you
get with the ThreadAnalyzer.  If you have the MT Debugger and SC3.0.1,
then, you should also have a copy of the ThreadAnalyzer (it was first
shipped on the same CD that had SC3.0.1) Look for the binary "tha"
under /opt/SUNWspro/bin.  

The "Collector" feature that you can use from inside the Debugger
doesn't work with MT programs.  Future MT-aware-profiling tools will
be integrated with the Debugger - is that where you'd like to use
profiling?
================================================================
 Q34: Sometimes the specified sleep time is SMALLER than what I want.  

>I have a program that generates UDP datagrams at regular intervals.
>It uses real time scheduling for improved accuracy.
>(The code I used is from the Solaris realtime manual.)
>
>This helps, but once in a while I do not get the delay I wanted.
>The specified sleep time is SMALLER (i.e. faster) than what I want.
>
>I use the following procedure for microsecond delays
>
>void
>delay(int us) /* delay in microseconds */
>{
>	struct timeval tv;
>
>	tv.tv_sec = us / 1000000;
>	tv.tv_usec = us % 1000000;
>	(void)select( 0, (fd_set *)NULL, (fd_set *)NULL, (fd_set *)NULL, &tv );
>
>}
>
>
>As I said, when I select a delay, occasionally I get a much smaller delay.
>
>examples:
>	Wanted: 19,776 microseconds, got: 10,379 microseconds
>	Wanted:    910 microseconds, got:    183 microseconds
>
>
>As you can see, the error is significant when it happens.
>It does not happen often. (0.5% of the time)
>
>I could use the usleep() function, but that's in the UCB library.
>Anyone have any advice?

A:

First of all, you can not do a sleep implementation in any increments
other than 10 milliseconds (or 1/HZ variable).

Second, there is a bug in the scheduler (fixed in 2.5) that may
mess up your scheduling in about 1 schedules around every
300,000 or so. 

Third, A much better timing interface will be available in
Solaris 2.6 (or maybe  earlier) thru posix interfaces. That
should give you microsecond resolution with less than 
50 microseconds latency.

Sinan
================================================================
 Q35: Any debugger that single step a thread while the others are running?  

|>  Has anyone looked into the possibility of doing a MT debugger
|> that will allow you to single step a thread while the others
|> are running? This will probably require a debugger that attaches
|> a debugger thread to each thread...

A:

This was the topic of my master's thesis. You might check:

http://plg.uwaterloo.ca/~mkarsten

and follow the link to the abstract or the full version.

Martin
	==================================================

We have used breakpoint debugging to debug threads programs. We have
implemented a debugger that enables the user to write scripts to debug
programs (not limited to threads programs). This is made possible by a Tcl
interface atop gdb and hooks in gdb, that exports some basic debugger
internals to the user domain.  Thus allowing the user to essentially write
his own Application Specific debugger.

Please see the following web page for more info on the debugger

http://www.eecs.ukans.edu/~halbhavi/debugger.html
or
http://www.tisl.ukans.edu/~halbhavi/debugger.html

Cheers
Sudhir Halbhavi
halbhavi@eecs.ukans.edu
================================================================
 Q36: Any DOS threads libraries?  

> Is there any way or does anyone have a library that will allow to program
> multitreads.. I need it for SVGA mouse functions.. I use both C++ and
> Watcom C++, 

A:

I use DesqView for my DOS based multi-thread programs.  (Only they don't call
them threads, they call them tasks....)  I like the DesqView interface to 
threads better than the POSIX/Solaris interface, but putting up with DOS was
almost more than I could stand.
================================================================
 Q37: Any Pthreads for Linux?  

A:

The is the *final* beta release of PCthreads (tm), base-level version 0.9.2.
This package is intended to provide POSIX .1c multithreading for Linux
developers wishing to develop applications requiring multiple threads of
execution.  PCthreads (tm) is catagorized as a "user-mode" thread package,
because it does not rely nor depend upon the existance of kernel-mode threads.

The release kit is now available and may retreived from

        sunsite.unc.edu /pub/Linux/devel/lang/c
        tsx-11.mit.edu /pub/linux/ALPHA/pthread

via anonymous FTP.  The kits were uploaded last night, 25 November and so may
take a day or two to appear.  Please replace your previous versions with this
one.
   +-----------------------------------------------------------------------+
   | Michael T. Peterson  | For the latest status on DCE and POSIX threads |
   | mtp@big.aa.net       | for Linux, see http://www.aa.net/~mtp/         |
   +-----------------------------------------------------------------------+


There is currently no kernel support for threads in LINUX.  There are,
however, two user level implementations of the POSIX-threads standard
available.

One can be found at
  ftp://ftp.cs.fsu.edu/pub/PART/PTHREAD/pthreads.tar.gz

This one was originally designed to run on SPARCs, but works on LINUX as
well (might need adaptions of the headerfiles, depending on the kernel
version).

The second one has been integrated into the libc package (get version 5.3.12
or newer. NOTE: The pthread-library has to be built separately!  See the
included README).

Both implementations only implement part of the POSIX standard.  As thread
switching occurs only when calling any of the thread library functions (as
far as I know), it appears to be safe to use the regular C/C++-libraries. I
would be glad on receiving further information about this topic, though :-)!


================================================================
 Q38: Any really basic C code example(s) and get us newbies started?  

>Could one of you threads gods please post some really, really basic C code
>example(s) and get us newbies started?  There just doesn't seem to be any other
>way for us to learn how to program using threads.

A:

The following is a compilation of all the generous help that was posted or mailed to me 
concerning the use of threads in introductory programs.  I apologize for it not being
edited very well...  (Now I just need time to go through all of these)

Here's all of the URL's:

http://www.pilgrim.umass.edu/pub/osf_dce/contrib/contrib.html
http://www.sun.com/workshop/threads
http://www.Sun.COM/smi/ssoftpress/catalog/books_comingsoon.html
http://www.aa.net/~mtp/


--Carroll
================================================================
 Q39: Please put some Ada references in the FAQ.  

A:

Most Ada books will introduce threading concepts.  Also, check out Windows
Tech Journal, Nov. 95 for more info on this subject.
================================================================
 Q40: Which signals are synchronous, and whicn are are asynchronous?  

>I have another question. Since we must clearly distinguish the
>sinchronous signals from the asynchronous ones for MT, is there any
>documentation on which is which? I could not find any.

A:

In general, independent of MT, this is an often mis-understood area of
signals.  The adjective: "synchronous"/"asynchronous" cannot be applied to a
signal.  This is because any signal (including normally synchronously
generated signals such as SIGSEGV) could be asynchronously generated using
kill(2), _lwp_kill(2) or thr_kill(3t).

e.g. SIGSEGV, which is normally synchronously generated, can also be sent
via kill(pid, SIGSEGV), in which case it is asynchronously generated. So
labelling SIGSEGV as synchronous and a program that assumes this, would be
incorrect.

For MT, a question is: would a thread that caused the generation of a signal
get this signal?

If this is posed for a trap (SIGSEGV, SIGBUS, SIGILL, etc.), the answer is:
yes - the thread that caused the trap would get the signal.  But the handler
for the trap signal, i.e. a SIGSEGV handler, for example, cannot assume that
the handler was invoked for a synchronously generated SIGSEGV (unless the
application knows that it could not have receieved a SIGSEGV via a kill(),
or thr_kill()).

If this question is posed for any other signal (such as SIGPIPE, or the
real-time signals) the answer should not really matter since the program
should not depend on whether or not the thread that caused the signal to be
generated, receives it. For traps, it does matter, but for any other signal,
it should not matter.

FYI: On 2.4 and earlier releases, SIGPIPE, and some other signals were sent
to the thread that resulted in the generation of the signal, but on 2.5, any
thread may get the signal. The only signals that are guaranteed to be sent
to the thread that resulted in its generation, are the traps (SIGILL,
SIGTRAP, SIGSEGV, SIGBUS, SIGFPE, etc.). This change should not matter since
a correctly written MT application would not depend on the synchronicity of
the signal generation for non-traps, given the above description of signal
synchronicity that has always been true.

-Devang
================================================================
 Q41: If we compile -D_REENTRANT, but without -lthread, will we have problems?  

>Hi -
>
>I had posed a question here a few weeks ago and received a response. Since
>then the customer had some follow-on questions. Can anyone address this
>customer's questions:
>
>(note: '>' refers to previous answer we provided customer)
>
>> If only mutexes are needed to make the library mt-safe, the library writer 
>> can do the following to enable a single mt-safe library to be used by both 
>> MT and single-threaded programs:
>
>Actually, we are only using the *_r(3c) functions, such as strtok_r(3c),
>getlogin_r(3c), and ctime_r(3c).  We are not actually calling thr_*,
>mutex_*, cond_*, etc. in the libraries.
>
>We want to use these *_r(3c) library functions instead of the normal
>non-MT safe versions (such as strtok(), ctime(), etc.), but if we compile
>the object files with -D_REENTRANT, but do not link with -lthread, will
>we have problems?

A:


No - you will not have any problems, if you do not link with -lthread.

But if your library is linked into a program which uses -lthread, then:

You might have problems in a threaded program because of how you allocate 
and use the buffers that are passed in to the *_r routines.

The usage of the *_r routines has to be thread-safe, or re-entrant in
the library. The *_r routines take a buffer as an argument. If the library
uses a global buffer to be passed to these routines, and does not protect
this buffer appropriately, the library would be unsafe in a threaded program.

Note that here, the customer's library has to do one of the following to ensure
that their usage of these buffers is re-entrant:

- if possible, allocate the buffers off the stack - this would be per-thread
  storage and would not require the library to do different things depending
  on whether the library is linked into a threaded program or not.

- if the above is not possible:

On any Solaris release, the following may be done: (recommended solution):

	- use mutexes, assuming that threads are present, to protect the 
	  buffers. If the threads library is not linked in, there are dummy 
	  entry points in libc for mutexes which do nothing - and so this 
	  will compile correctly and still work. If the threads library is 
	  linked in, the mutexes will be real and the buffers will be 
	  appropriately protected.

On Solaris 2.5 only:

	- if you do not want to use mutexes for some reason and want to use
	  thread-specific data (TSD) if threads are present (say), then on 2.4
          you cannot do anything. On 2.5, though, one of the following may be 
	  done:
 
	(a) on 2.5, you can use thr_main() to detect if threads are linked in 
          or not. If they are, carry out appropriate TSD allocation of buffers.

	(b) If you are sure only POSIX threads will be used (if at all), and you
	  do not like the non-portability of thr_main() which is not a POSIX
	  interface, then, on 2.5, you can use the following (hack) to detect if
	  pthreads are linked in or not: you need the #pragma weak declaration 
	  so that you can check if a pthreads symbol is present or not. If 
	  it is, then pthreads are linked in, otherwise they are not. Following
	  is a code snippet which demonstrates this. You can compile it with
	  both -lpthread and without. If compiled without -lpthread it prints
	  out the first print statement. If compiled with -lpthread, it prints
	  out the second print statement. I am not sure if this usage of
	  #pragma weak is any more portable than using thr_main().

		#include 

		#pragma weak pthread_create

		main()
		{
			if (pthread_create == 0) {
				printf("libpthread not linked\n");
			} else {
				printf("libpthread is present\n");
				/*
				 * In this case, use Thread Specific Data
				 * or mutexes to protect access to the global
				 * buffers passed to the *_r routines.
				 */
			}
		}




-Devang

================================================================
 Q42: Can Borland C++ for OS/2 give up a TimeSlice?  

Johan>    Does anyone know if Borland C++ for OS/2 has a function that could be 
Johan>    used within a THREAD to give up a TimeSlice.

A:

	If all you want to do is give up your timeslice
		DosSleep(0)
however if you are the highest priority thread, you will be immediately dispatched
again, before other threads.  Even when all the threads are the same priority,
my understanding is that the OS/2 operating system has a degradation algorithm
for the threads in a process ... so even if you DosSleep with the "same" priority
your thread still could be dispatched immediately --- depending on the
degradation algorithm.

	If you want to sleep to next clock tick
		DosSleep(1)
works, because the system round the 1 up to the next clock tick value.
This should allow other threads in your process to be dispatched.

	Both are valid semantics, depending on what you would prefer.
--
William E. Hannon Jr.                         internet:whannon@austin.ibm.com
DCE Threads Development                                        whannon@austin
================================================================
 Q43: Are there any VALID uses of suspension?  

    UI threads, OS/2 and NT all allow you to suspend a thread.  I have yet to
  see a program which does not go below the API (ie debuggers, GCs, etc.), but
  still uses suspension.  I don't BELIEVE there is a valid use.  I could be
  wrong.

A:

I'll bite.  Whether we "go below the API" or not is for you to decide.
Our product, ObjectStore, is a client-server object-oriented database
system.  For the purpose of this discussion, it functions like a
user-mode virtual memory system: We take a chunk of address space
and use it as a window onto a database; if the user touches an address
within our special range, we catch the page fault, figure out which
database page "belongs" there, and read that page from the server.  After
putting the page into place, we continue the faulting instruction, which
now succeeds, and the user's code need never know that it wasn't there
all the time.

This is all fine for a single-threaded application.  There's a potential
problem for MT applications, however; consider reading a page from a
read-only database.  Thread A comes along and reads a non-existent page.
It faults, the fault enters our handler, and we do the following:
	get data from server
	make page read-write	;open window
	copy data to page
	make page read-only	;close window
During the window between the two page operations, another thread can
come along and read invalid data from the page, or in fact write the
page, with potentially disastrous effect.

On Windows and OS/2, we do the following:
	get data from server
	suspend all other threads
	make page read-write
	copy data to page
	make page read-only
	resume all other threads
to prevent the "window" from opening.  On OS/2, we use DosEnterCritSec,
which suspends all other threads.  On NT, we use the DllMain routine
to keep track of all the threads in the app, and we call SuspendThread
on each.  We're very careful to keep the interval during which threads
are suspended as brief as possible, and on OS/2 we're careful not to call
the C runtime while holding the critical section.

On most Unix systems, we don't have to do this, because mmap() has the
flexibility to map a single physical page into two or more separate
places in the address space.  This enables us to do this:
	get data from server
	make read-write alias of page, hidden from user
	copy data to alias page
	make read-only page visible to user
The last operation here is atomic, so there's no opportunity for other
threads to see bogus data.  There's no equivalent atomic operation on
NT or OS/2, at least not one that will operate at page granularity.
	===============================================
Since you do not like Suspend/Resume to be available to user level apis,
I thought the following set of functions (available to programs)
in WinNT (Win32) might catch your interest :) :

CreateRemoteThread -- allows you to start a thread in another process's
address space.. The other process may not even know you've done it
(depending on circumstances).  Supposedly, with full security turned
on (off by default!) this won't violatge C2 security.

SetThreadContext/GetThreadContext - Just lke it sounds.  You can
manipulate a thread's context (CPU registers, selectors, etc!).

Also, you can forcibly map a library (2-3 different ways: createremotethread
can allow this as well) to another proces's address space (that is, you
can map a DLL of yours to a running process).  Then, you can do
things like spawn off threads, after you have invisibly mapped your DLL
into the space.  Yes, there is potential for abuse (and for interestiing
programs).

But, microsoft has a use for these things.  They can help you subclass
a window on the desktop for instance.  If you wanted to make say
Netscape's main window beep twice every time it repaints, you could
map a DLL into netscape's address space, subclass the main window
(subclass == "Send ME the window's messages, instead of sending it to
the window -- i'll take care of everything!"), and watch for PAINTs
to come through.

Anyway, don't mean to waste your time.  Just thought you might find
it interesting that a user can start additional threads in someone else's
process, change thread context forcibly (to a decent degree), and
even latch onto a running process in order to change its behavior, or
just latch on period to run a thread you wrote in another proceses's
address space.




================================================================
 Q44: What's the status of pthreads on SGI machines?  
>> We are considering porting of large application from Concurrent Computer
>> simmetrical multiprocessor running RTU-6 to one of the Silicon Graphics
>> multiprocessors running IRIX (5.3?).
>> 
>> Our application uses threads heavily. Both so-called user threads and 
>> kernel threads are required with a fair level of synchronization 
>> primiteves support and such.
>> 
>> My question is: what kind of multi-threaded application programming 
>> support is available in IRIX? 
>> 
>> Reading some of the SGI technical papers available on their WWW page 
>> just confuses me. I know that Posix threads or Solaris-type 
>> LWP/threads supports would be OK. 

A:

POSIX thread support in IRIX is more than a rumor - pthreads are currently 
scheduled to be available shortly after release of IRIX 6.2 (IRIX 6.2 is 
currently scheduled for release in Feb 96).  If you are interested in 
obtaining pthreads under IRIX as soon as possible, I would recommend 
contacting your local SGI office.
-- 
Bruce Johnson, SGI ASD                 
Real-time Applications Engineering          
================================================================
 Q45: Does the Gnu debugger support threads?  

A:

An engineer at Cygnus is implementing thread support in gdb for Solaris.
No date for completion is given.
================================================================
 Q46: What is gang scheduling?  

A:

Gang Scheduling is described a variety of ways. Generally the
consistent thread is that a GS gives a process all the processors at
the same time (or none for a time slice). This is most helpful for
"scientific apps" because the most common set up is something like

	do i=1, bignum
	   stuff
	   more stuff
	   lots more stuff
	end do

the obvious decomposition is bignum/nproc statically allocated. Stuff
and friends take very close to the same time per chunk, so if you get
lucky it all happens in one chime (viz. one big clock). Else it takes
precisely N chimes with no leftovers. When unlucky, it's N chimes +
cleanup for stragglers.

Virtually all supercomputers do this, they may not even bother to give
it a special name. SGI makes this explicit (and supported).

On SPARC/Solaris there is no way for the compiler to know if we'll get
the processors requested or when. So you can suffer multiple chime
losses quite easily.

One can reallocate processor/code on the fly, but with increased overhead.
================================================================
 Q47: LinuxThreads linked with X11, calls to X11 seg fault. 


You can't rely on libraries that are not at the very least compiled
with -DREENTRANT to do anything reasonable with threads.  A vanilla
X11 build (with out -DREENTRANT and without XTHREADS enabled) 
will likely behave badly with threads.  

It's not terribly hard to build X with thread support these days,
especially if you're using libc-6 with builtin LinuxThreads.  Contact
your Linux distribution maintainer and insist on it.  Debian has just 
switched to a thread-enabled X11 for their libc6 release; has any other
distribution? 

Bill Gribble
================================================================
 Q48: Are there Pthreads on Win32?  

Several answers here.  #1 is probably the best (most recent!).


A: Yes, there is a GNU pthreads library for Win32.  It is still under
   active development, but you can find out more by looking at
   http://sourceware.cygnus.com/pthreads-win32/

   (This is a combination of Ben Elliston & John Bossom's work. & others?)


Also:

Well, Dave Butenhof will probably kill me for saying this, but Digital has a
pthreads implementation for WIN32. I bug them occasionally about packaging
up the header and dll and selling it separately (for a reasonable price, of
course). I think it's a great idea. My company has products on NT and UNIX,
so it would solve some painful portability issues for us.  This
implementation uses the same "threads engine" that Digital uses, rather

than just some wrappers on NT system services.

So, maybe if a few potential customers join me in asking Digital for this,
we'll get somewhere.  What say, Dave?

================

I have such a beast...sort of.

I have a pthreads draft 4 wrapper that is (nearly) complete and has been
in use for a while (so it seems to work!).

About 6 weeks back I changed this code to provide a draft 10 interface. This
code has however not yet been fully tested nor folded into my projects.
Casting my mind back (a lot has happened in 6 weeks!) I seem to remember
one or two small issues where I wasn't sure of the semantics; I was working
from a document I picked up at my last job which showed how to migrate
from pthreads 4 to pthreads 10, rather than a copy of the standard.

If anyone wants this code, I can make it available.

Ian
ian.johnston@ubs.com

		================
> > As far as I know, there is no pthreads implementation for NT.  However,
> > ACE provides a C++ threads wrapper which works on pthreads, and on NT
> > (and some others).
>
> Well, Dave Butenhof will probably kill me for saying this, but Digital has a
> pthreads implementation for WIN32. I bug them occasionally about packaging up
> the header and dll and selling it separately (for a reasonable price, of
> course). I think it's a great idea. My company has products on NT and UNIX,
> so it would solve some painful portability issues for us. This implementation
> uses the same "threads engine" that Digital uses, rather than just some
> wrappers on NT system services.

Yes, DECthreads has been ported to Win32 for quite a while. It runs on Windows
NT 3.51 and 4.0, on Alpha and Intel; and also on Windows 95 (though this was
not quite as trivial as Microsoft might wish us to believe.)

The main questions are:

  1. What's the market?
  2. How do we distribute the code, and at what cost? (Not so much "cost to the
     customer", as "cost to Digital".)

The big issue is probably that, especially with existing free code such as ACE,
it seems unlikely that there'd be much interest unless it was free or "dirt
cheap". Yet, even if we disclaim support, there will still be costs associated,
which means it'd be really tricky to avoid losing money.

> So, maybe if a few potential customers join me in asking Digital for this,
> we'll get somewhere.  What say, Dave?

We'd love to hear who wants this and why. Although I haven't felt comfortable
actually advertising the possibility here, I have forwarded the requests I've
seen here, and recieved via mail (including Jeff's) to my manager, who is the
first (probably of several) who needs to make any decisions.

I'd be glad to forward additional requests. Indications of what sort of product
(e.g., in particular, things like "sold for profit" or "internal utility"
distinctions), and, of course, whether (and how much) you'd be willing to pay,
would be valuable information.

/---------------------------[ Dave Butenhof ]--------------------------\

From: Ben Elliston 

Matthias Block  writes:

> is there someone who knows anything about a Pthread like library for
> Windows NT. It would simplify the work here for me.

I am involved with a free software project to implement POSIX threads
on top of Win32.   For the most part, it is complete, but it's still
well and truly in alpha testing right now.

I expect to be posting an announcement in a few weeks (say, 4?) to
comp.programming.threads.  The source code will be made available via
anonymous CVS for those who want to keep up to date or submit
patches.  I'm looking forward to getting some net testing!


Over the last several months I have seen some requests for
a Win32 implementation of PThreads.  I, too, had been looking
for such an implementation but to no avail.

Back in March, I decided to write my own. It is based upon the
PThreads 1003.1c standard, however, I didn't implement everything.
Missing is signal handling and real-time priority functions.

I based the implementation on the description provided by

    Programming with POSIX Threads, by
    Dave R. Butenhof

I've created a zipped file consisting of some header files, an implib, 
a DLL and a simple test program.

I'm providing this implementation for free and as-is. You may download it
from

    http://www.cyberus.ca/~jebossom/pthread1c.html

Cheers,

John


--
John E. Bossom                                     Cognos Inc.
Voice: (613) 738-1338 x3386        O_o             P.O. Box 9707, Stn. T
  FAX: (613) 738-0002             =(  )= Ack!      OTTAWA, ON  K1G 4K9
 INET:  jebossom@cognos.COM          U             CANADA
================================================================
 Q49: What about garbage collection?  


Please, please, please mention garbage collection when you come around
to talking about making code multithreaded.  A whole lot of
heap-allocated data needs to be explicitly reference counted *even
more* in a multithreaded program than in a single threaded program
(since it is so much harder to determine whether data is live or not),
and this leads to lots of bugs and worries and nonsense.

With garbage collection, on the other hand, you get to throw away
*all* of your worries over memory management.  This is a tremendous
win when your brain is already approaching meltdown due to the strain
of debugging subtle race conditions.

In addition, garbage collection can help to make the prepackaged
libraries you link against safer to play with (although it obviously
won't help to make them thread safe).  Xt, for example, is very badly
written and leaks like a sieve, but a conservative garbage collector
will safely kill off those memory leaks.  If you're linking against
legacy libraries and you need to write a long-running multithreaded
server, GC can make the difference between buying more RAM and killing
your server every few days so that it doesn't thrash, and simply
plugging in the threads-aware GC and sailing fairly happily along.

Bryan O'Sullivan 

[Please see: Geodesic Systems (www.geodesic.com)     -Bil]

================================================================
 Q50: Does anyone have any information on thread programming for VMS?  

No ftp or web stuff, although we do have an HTML version of the Guide to
DECthreads and we'll probably try to get it outside the firewall where
it'll do y'all some good, one of these days. I've been very impressed
with Sun's "thread web site", and I'd like to get Digital moving in that
direction to help with the global work of evangelizing threads... but
not until I've finished coding, and writing, and consulting, and all
sorts of other things that seem to take 500% of my time. For general
info, and some information (though not enough) on using POSIX threads,
check Sun's library. (They need to start tapering off the UI threads.)

If you've got VMS (anything since 5.5-2), you'll have a hardcopy of the
Guide in your docset, and on the doc cdrom in Bookreader format. OpenVMS
version 7.0 has POSIX 1003.1c-1995 threads -- anything earlier has only
the old CMA and 1003.4a/D4 "DCE threads". Furthermore, OpenVMS Alpha 7.0
supports SMP threads (kernel support for dynamic "many to few"
scheduling), although "binary compatibility paranoia" has set in and it
may end up being nearly impossible to use. OpenVMS VAX 7.0 does not have
SMP or kernel integration -- integration will probably happen "soon",
but VAX will probably never have SMP threads.

------------------------------------------------------------------------
Dave Butenhof                              Digital Equipment Corporation
butenhof@zko.dec.com                       110 Spit Brook Rd, ZKO2-3/Q18
Phone: 603.881.2218, FAX: 603.881.0120     Nashua, NH 03062-2711
                 "Better Living Through Concurrency"
------------------------------------------------------------------------
================================================================
 Q51: Any information on the DCE threads library?  

 http://www.osf.org/dce/
================================================================
 Q52: Can I implement pthread_cleanup_push without a macro?  

I was about to use pthread_cleanup_push, when I noticed that it is
implemented as a macro (on Solraris 2.5) which forces you to have the
pthread_cleanup_pop in the same function by having an open brace { at the
end of the first macro and closing it int the second...  Since I want to
hide most of this stuff in something like a monitor (or a guard in ACE) in
C++ by using the push in a constructor and the pop in the destructor I'm
wondering if there is something fondamental that would prevent me to do so
or could I just re-implement the stuff done by the macros inside some class
services.



POSIX 1003.1c-1995 specifies that pthread_cleanup_push and pthread_cleanup_pop
must be used at the same lexical scope, "as if" the former were a macro that
expands to include an opening brace ("{") and the latter were a macro that
expands to include the matching closing brace ("}").

The Solaris 2.5 definition therefore conforms quite accurately to the intent
of the standard. And so does the Digital UNIX definition, for that matter. If
you can get away with "reverse engineering" the contents of the macros, swell;
but beware that this would NOT be a service to those using your C++ package,
as the results will be extremely non-portable. In fact, no guarantees that it
would work on later versions of Solaris, even assuming strict binary
compatibility in their implementation -- because they could reasonably make
"compatible" changes that would take advantage of various assumptions
regarding how those macros are used that you would be violating.

What you want to do has merit, but you have to remember that you're writing in
C++, not C. The pthread_cleanup_push and pthread_cleanup_pop macros are the C
language binding to the POSIX 1003.1c cancellation cleanup capability. In C++,
the correct implementation of this capability is already built into the
language... destructors. That is, C++ and threads should be working together
to ensure that C++ destructors are run when a thread is cancelled. If that is
done, you've got no problem. If it's not done, you've got far worse problems
anyway since you won't be "destructing" most of your objects anyway.

/---[ Dave Butenhof ]-----------------------[ butenhof@zko.dec.com ]---\
| Digital Equipment Corporation           110 Spit Brook Rd ZKO2-3/Q18 |
| 603.881.2218, FAX 603.881.0120                  Nashua NH 03062-2698 |
\-----------------[ Better Living Through Concurrency ]----------------/


================================================================
 Q53: What switches should be passed to particular compilers?  

> Does anyone have a list of what switches should be passed to particular
> compilers to have them generate thread-safe code?  For example,
> 
> Solaris-2 & SunPro cc       : -D_REENTRANT
> Solaris-2 & gcc             : ??
> DEC Alpha OSF 3.2 & /bin/cc : -threads
> IRIX 5.x & /bin/cc          : ??
> 
> Similarly, what libraries are passed to the linker to link in threads
> support?
> 
> Solaris-2 & Solaris threads : -lthread
> DEC Alpha OSF 3.2 threads   : -lpthreads
> IRIX 5.x & IRIX threads     : (none)
> 
> And so forth.
> 
> I'm trying to get GNU autoconf to handle threads gracefully.
> 
> Bill

That would be useful information in general, I suppose. I can supply the
information for Digital UNIX (the operating system previously known as
"DEC OSF/1"), at least.

For 3.x and earlier, the proper compiler switch is -threads, which (for
cc) is effectively just -D_REENTRANT. For linking, the cc driver expands
-threads to "-lpthreads -lmach -lc_r" -- you need all three, immediately
preceeding -lc (which must be at the end). -lpthreads isn't enough, it
will pull in libmach and libc_r implicitly and in the wrong order (after
libc, where they will fail to preempt symbols).

For 4.0, you can still use -threads if you're using the DCE threads (D4)
or cma interfaces. If you don't use -threads, the link libraries should
be changed to "-lpthreads -lpthread -lmach -lexc" (before -lc). If you
use 1003.1c-1995 threads, you use "-pthread" instead of "-threads". cc
still gets -D_REENTRANT, but ld gets -lpthread -lmach -lexc.

/---[ Dave Butenhof ]-----------------------[ butenhof@zko.dec.com ]---\
| Digital Equipment Corporation           110 Spit Brook Rd ZKO2-3/Q18 |
| 603.881.2218, FAX 603.881.0120                  Nashua NH 03062-2698 |
\-----------------[ Better Living Through Concurrency ]----------------/


================================================================
 Q54: How do I find Sun's bug database?  

>I am trying to use Thread Analyzer in Solaris 2.4 for performance
>tuning. But after loading the trace directory, tha exit with following
>error message: 
>Thread Analyzer Fatal Error[0]: Slave communication failure


It always helps if you state which version of the application you are
using, in this case the Thread Analyzer.

There have been a number of bugs which result in this error message
that have been fixed.  Please obtain the latest ThreadAnalyzer patch
from your Authorized Service Provider (ASP) or from our Wep page:

http://access1.sun.com/recpatches/DevPro.html
================================================================
 Q55:  How do the various vendors' threads libraries compare?  

    Fundamentally, they are all based on the same paradigm, and everything
    you can do in one library you can (pretty much) do in any other.  Ease
    of programming and efficency will be the major distinctions.

OS                Preferred Threads POSIX Version   Kernel Support Sched model
---------------   ----------------- -------------   -------------- -------------
Solaris 2.5       UI-threads        1003.1c-1995    yes            2 level(1)
SVR4.2MP/UW 2.0   UI-threads        No
IRIX 6.1          sproc             No
IRIX 6.2          sproc             1003.1c-1995(2)
Digital UNIX 3.2  cma               Draft 4         yes            1 to 1
Digital UNIX 4.0  1003.1c-1995      1003.1c-1995    yes            2 level
DGUX 5.4          ?                 Draft 6         yes
NEXTSTEP          (cthreads?)       No
AIX 4.1           AIX Threads(3)    Draft 7         yes            1 to 1
Plan 9            rfork()           No
OpenVMS 6.2       cma               Draft 4         no
OpenVMS Alpha 7.0 1003.1c-1995      1003.1c-1995    yes            2 level
OpenVMS VAX 7.0   1003.1c-1995      1003.1c-1995    no
WinNT             Win32 threads     No
OS/2              DosCreateThread() Draft 4
Win32             Win32 threads     No              yes            1 to 1

Notes:

1) Solaris 2.5 blocks threads in kernel with LWP, but provides a signal to
allow user level scheduler to create a new LWP if desired (and
thr_setconcurrency() can create additional LWPs to minimize the chances of
losing concurrency due to blocking.)

2) According to IRIX 6.2 info on SGI's web, 1003.1c-1995 threads will be
provided only as part of the REACT/pro 3.0 Realtime Extensions kit, not in
the base O/S.

3) Can anyone clarify this? My impression is that AIX 4.1 favors 1003.4a/D7
threads; but then I've never heard the term "AIX Threads".

================================================================
 Q56: Why don't I need to declare shared variables VOLATILE?  


> I'm concerned, however, about cases where both the compiler and the
> threads library fulfill their respective specifications.  A conforming
> C compiler can globally allocate some shared (nonvolatile) variable to
> a register that gets saved and restored as the CPU gets passed from
> thread to thread.  Each thread will have it's own private value for
> this shared variable, which is not what we want from a shared
> variable.

In some sense this is true, if the compiler knows enough about the
respective scopes of the variable and the pthread_cond_wait (or
pthread_mutex_lock) functions. In practice, most compilers will not try
to keep register copies of global data across a call to an external
function, because it's too hard to know whether the routine might
somehow have access to the address of the data.

So yes, it's true that a compiler that conforms strictly (but very
aggressively) to ANSI C might not work with multiple threads without
volatile. But someone had better fix it. Because any SYSTEM (that is,
pragmatically, a combination of kernel, libraries, and C compiler) that
does not provide the POSIX memory coherency guarantees does not CONFORM
to the POSIX standard. Period. The system CANNOT require you to use
volatile on shared variables for correct behavior, because POSIX
requires only that the POSIX synchronization functions are necessary.

So if your program breaks because you didn't use volatile, that's a BUG.
It may not be a bug in C, or a bug in the threads library, or a bug in
the kernel. But it's a SYSTEM bug, and one or more of those components
will have to work to fix it.

You don't want to use volatile, because, on any system where it makes
any difference, it will be vastly more expensive than a proper
nonvolatile variable. (ANSI C requires "sequence points" for volatile
variables at each expression, whereas POSIX requires them only at
synchronization operations -- a compute-intensive threaded application
will see substantially more memory activity using volatile, and, after
all, it's the memory activity that really slows you down.)

/---[ Dave Butenhof ]-----------------------[ butenhof@zko.dec.com ]---\
| Digital Equipment Corporation           110 Spit Brook Rd ZKO2-3/Q18 |
| 603.881.2218, FAX 603.881.0120                  Nashua NH 03062-2698 |
\-----------------[ Better Living Through Concurrency ]----------------/

================================================================
 Q57: Do pthread_cleanup_push/pop HAVE to be macros (thus lexically scoped)?  

Paul Pelletier wrote:
 
I was about to use pthread_cleanup_push, when I noticed that it is
implemented as a macro (on Solaris 2.5) which forces you to have the
pthread_cleanup_pop in the same function by having an open brace { at the
end of the first macro and closing it int the second...  Since I want to
hide most of this stuff in something like a monitor (or a guard in ACE) in
C++ by using the push in a constructor and the pop in the destructor I'm
wondering if there is something fundamental that would prevent me to do so
or could I just re-implement the stuff done by the macros inside some class
services.
 

POSIX 1003.1c-1995 specifies that pthread_cleanup_push and
pthread_cleanup_pop must be used at the same lexical scope, "as if" the
former were a macro that expands to include an opening brace ("{") and the
latter were a macro that expands to include the matching closing brace
("}").

The Solaris 2.5 definition therefore conforms quite accurately to the intent
of the standard. And so does the Digital UNIX definition, for that
matter. If you can get away with "reverse engineering" the contents of the
macros, swell; but beware that this would NOT be a service to those using
your C++ package, as the results will be extremely non-portable. In fact, no
guarantees that it would work on later versions of Solaris, even assuming
strict binary compatibility in their implementation -- because they could
reasonably make "compatible" changes that would take advantage of various
assumptions regarding how those macros are used that you would be violating.

What you want to do has merit, but you have to remember that you're writing
in C++, not C. The pthread_cleanup_push and pthread_cleanup_pop macros are
the C language binding to the POSIX 1003.1c cancellation cleanup
capability. In C++, the correct implementation of this capability is already
built into the language... destructors. That is, C++ and threads should be
working together to ensure that C++ destructors are run when a thread is
cancelled. If that is done, you've got no problem. If it's not done, you've
got far worse problems anyway since you won't be "destructing" most of your
objects anyway.

/---[ Dave Butenhof ]-----------------------[ butenhof@zko.dec.com ]---\

================================================================
 Q58: Thread Analyzer Fatal Error[0]: Slave communication failure ??  

>I am trying to use Thread Analyzer in Solaris 2.4 for performance
>tuning. But after loading the trace directory, tha exit with following
>error message: 
>Thread Analyzer Fatal Error[0]: Slave communication failure
>
>I do not know what happened. 

It always helps if you state which version of the application you are
using, in this case the Thread Analyzer.

There have been a number of bugs which result in this error message
that have been fixed.  Please obtain the latest ThreadAnalyzer patch
from your Authorized Service Provider (ASP) or from our Wep page:

http://access1.sun.com/recpatches/DevPro.html

Chuck Fisher 

================================================================
 Q59: What is the status of Linux threads?  


Linux has kernel-level threads now and has had a thread-safe libc for a
while.  With LinuxThreads, you don't have to worry about things like your
errno, or blocking system calls. The few standard libc functions that are
inherently not thread safe (due to using static data areas) have been
augmented with thread-safe alternatives.

LinuxThreads are not POSIX, however.

http://sunsite.unc.edu/pub/Linux/docs/faqs/Threads-FAQ/html

================================================================
 Q60: The Sunsoft debugger won't recognize my PThreads program!  

Nope.  The 3.0.2 version was written before the release of Sun's pthread
library.  However, if you simply include -lthread on the compile line, it
will come up and work.  It's a little bit redundant, but works fine.  Hence:

%cc -o one one.c -lpthread -lthread -lposix4 -g

================================================================
 Q61: How are blocking syscall handled in a two-level system?  

> Martin Cracauer wrote:
> >
> > In a thread system that has both user threads and LWPs like Solaris,
> > how are blocking syscall handled?
> 
> Well, do you mean "like Solaris", or do you mean "Solaris"? There's no
> one answer for all systems. LWP, by the way, isn't a very general term.
> Lately I've been using the more cumbersome, but generic and relatively
> meaningful "kernel execution contexts". A process is a KEC, an LWP is a
> KEC, a "virtual processor" is a KEC, a Mach thread is a KEC, an IRIX
> sproc is a KEC, etc.
> 
> > By exchanging blocking syscalls to nonblocking like in a
> > pure-userlevel thread implementation?
> 
> Generally, only "pure user-mode" implementations, without any kernel
> support at all, resort to turning I/O into "nonblocking". It's just not
> an effective mechanism -- there are too many limitations to the UNIX
> nonblocking I/O model.
> 
> > Or by making sure a thread that calls a blocking syscall is on its own
> > LWP (the kernel is enterend anyway, so what would be the cost to do
> > so)?
> 
> Solaris 2.5 "latches" a user thread onto an LWP until it blocks in user
> mode -- on a mutex, a condition variable, or until it yields. User
> threads aren't timesliced, and they stick to the LWP across kernel
> blocks. If all LWPs in a process block in the kernel, a special signal
> allows the thread library to create a new one, but other than that you
> need to rely a lot on thr_setconcurrency.
> 
> Digital UNIX 4.0 works very differently. The kernel delivers "upcalls"
> to the user mode scheduler to communicate various state changes. User
> threads, for example, are timesliced on our KECs (which are a special
> form of Mach thread). When a thread blocks in the kernel, the user mode
> scheduler is informed so that a new user thread can be scheduled on the
> virtual processor immediately. The nice thing about this model is that
> we don't need anything like thr_setconcurrency to keep things running.
> Compute-bound user threads can't lock each other out unless one is
> SCHED_FIFO policy. And instead of "fixing things up" by adding a new
> kernel execution context when the last one blocks (giving you a
> concurrency level of 1), we keep you running at the maximum level of
> concurrency supportable -- the number of runnable user threads, or the
> number of physical processors, whichever is less.
> 
> Neither model (nor implementation) is perfect, and it would be safe to
> assume that both Digital and Sun are working on improving every aspect.
> The models may easily become very different in the future.
> 
> /---[ Dave Butenhof ]-----------------------[ butenhof@zko.dec.com ]---\
> | Digital Equipment Corporation           110 Spit Brook Rd ZKO2-3/Q18 |
> | 603.881.2218, FAX 603.881.0120                  Nashua NH 03062-2698 |
> \-----------------[ Better Living Through Concurrency ]----------------/

-- 
> Georges Brun-Cottan wrote:
> > So recursive mutex is far more than just a hack for lazy programmer or
> > just a way to incorporate non MT safe third party code. It is a tool
> > that you need in environment such OOP, where you can not or you do not
> > want to depend of an execution context.
> 
> Sorry, but I refuse to believe that good threaded design must end where
> OOP begins. There's no reason for two independently developed packages
> to share the same mutex. There's no reason for a package to be designed
> without awareness of where and when mutexes are locked. Therefore, in
> either case, recursive mutexes remain, at best, a convenience, and, at
> worst (and more commonly), a crutch.
> 
> I created the recursive mutex for DCE threads because we were dealing
> with a brand-new world of threading. We had no support from operating
> systems or other libraries. Hardly anything was "thread safe". The DCE
> thread "global mutex" allowed any thread-safe code to lock everything
> around a call to any unsafe code. As an intellectual exercise, I chose
> to implement the global mutex by demonstrating why we'd created the
> concept of "mutex attributes" -- previously, there had been none. As a
> result of this intellectual exercise, it became possible for anyone to
> conveniently create their own recursive mutex, which is locked and
> unlocked using the standard POSIX functions. There really wasn't any
> point to removing the attribute, since it's not that hard to create your
> own recursive mutex.
> 
> Remember that whenever you use recursive mutexes, you are losing
> performance -- recursive mutexes are more expensive to lock and unlock,
> even without mutex contention (and a recursive mutex created on top of
> POSIX thread synchronization is a lot more expensive than one using the
> mutex type attribute). You are also losing concurrency by keeping
> mutexes locked so long and across so much context that you become
> tempted to use recursive mutexes to deal with lock range conflicts.
> 
> Yes, it may be harder to avoid recursive mutexes. Although I've never
> yet seen a valid case proving that recursive mutexes are NECESSARY, I
> won't deny that there may be one or two. None of that changes the fact
> that an implementation avoiding recursive mutexes will perform, and
> scale, far better than one relying on recursive mutexes. If you're
> trying to take advantage of multithreading, all the extra effort in
> analysis and design will pay off in increased concurrency.
> 
> But, like any other aspect of performance analysis, you put the effort
> where the pay is big enough. There are non-critical areas of many
> libraries where avoiding recursive mutexes would be complicated and
> messy, and where the overhead of using them doesn't hurt performance
> significantly. Then, sure, use them. Just know what you're doing, and
> why.
> 
> /---[ Dave Butenhof ]-----------------------[ butenhof@zko.dec.com ]---\
> | Digital Equipment Corporation           110 Spit Brook Rd ZKO2-3/Q18 |
> | 603.881.2218, FAX 603.881.0120                  Nashua NH 03062-2698 |
> \-----------------[ Better Living Through Concurrency ]----------------/

================================================================
 Q62: Can one thread read from a socket while another thread writes to it?  

It's supposed to work!  That's certainly how sockets are defined.  It's
an easy enough test on your own system.
================================================================
 Q63: What's a good way of writing threaded C++ classes?  

> 
> Ian Emmons wrote:
> >
> > Baard Bugge wrote:
> > >
> > > >How would we put the whole object into a thread?
> > >
> > > Been there. Done that. Let the constructor create a thread before
> > > returning to the caller (another object). But beware, your OS will
> > > propably start the thread by calling a function (specified by you)
> > > C-style. You want this function to be a member function in your class,
> > > which is ok as long as you make it static. The thread function will
> > > also need the this-pointer to your newly created object. What you want
> > > will look something like this (in NT):
> > >
> > > // Thread callback function.
> > > // NOTE: Need to be written in C or be a static member function
> > > // because of C style calling convention (no hidden this pointer)
> > > LPTHREAD_START_ROUTINE CThread::ThreadFunc(LPVOID inputparam)
> > > {
> > >    CThread *pseudo_this = (CThread *) inputparam;
> > >    ...
> > > }
> > >
> > > This function have access to all the members in the object through the
> > > pseudo this pointer. And all member functions called by this function
> > > will run in the same thread. You'll have to figure out how to
> > > communicate with the other objects in your system though. Be careful.
> > >
> > > --
> > > BaBu
> >
> > You can take this even a step further.  Add a pure virtual to your generic
> > CThread class like so:
> >
> > class CThread
> > {
> >       ...
> > protected:
> >     // I don't remember what Win32 expects as the return value, here,
> >     // but you can fix this up as you wish:
> >     virtual unsigned entryPoint() = 0;
> >       ...
> > };
> >
> > Then have the static ThreadFunc call it like so:
> >
> > // Thread callback function.
> > // NOTE: Need to be written in C or be a static member function
> > // because of C style calling convention (no hidden this pointer)
> > LPTHREAD_START_ROUTINE CThread::ThreadFunc(LPVOID inputparam)
> > {
> >    return ((CThread*) inputparam)->entryPoint();
> > }
> >
> > Now, to create a specific thread, derive from CThread, override entryPoint,
> > and you no longer have to mess around with a pseudo-this pointer, because
> > the real this pointer is available.
> >
> > One tricky issue:  make sure you differentiate between methods that the
> > thread itself will call, and methods that other threads (such as the one
> > that created the thread object) will call -- you will need to do thread
> > synchronization on class members that are shared data.
> >
> > Ian
> >
> > ___________________________________________________________________________
> > Ian Emmons                                       Work phone: (415) 372-3623
> > ian@persistence.com                              Work fax:   (415) 341-8432
> > Persistence Software, 1720 S. Amphlett Blvd. Suite 300, San Mateo, CA 94402
> 

------------------
OK, let me warn everyone this is a very long response, but I just came off
of a large radar project on which I had to design multithreaded objects so
this question jumped out at me.

Yousuf Khan  wrote in article
<32CD2EAB.5678@bellglobal.com>...
> I got some hypothetical questions here, I'm not actually now trying to
> do any of this, but I can see myself attempting something in the near
> future.
> 
> Okay, I'm thinking multithreading and OO design methodologies are
> tailor-made for each other, in theory at least. OO design mandates that
> all object instances are considered concurrent with each other. That
> seems like a perfect application of threading principles. However,
> current threading protocols (such the POSIX Pthreads, Microsoft/IBM
> threads, Sun UI threads, etc.) seem to be based around getting
> subprocedures threaded, rather than getting objects threaded.

First, let me state my own programming background so you can apply the
appropriate grain of salt to what I say and understand my assumptions.  I
have programmed first for a few years in a DEC, VMS environment and then for
several more in a Windows/Windows NT environment.

> Okay, I suppose we can get individual methods within an object to be
> threaded, because they are just like subprocedures anyways. But what if we
> wanted to be extremely pedantic, and we want the entire object to be in
> its own thread, in order to be true to OO design paradigms? How would we
> put the whole object into a thread?  My feeling is that we should just
> call the object's constructor inside a thread wrapper, that way the entire
> object will go into a thread, including any other methods that are part of
> that object. What I guess I'm saying is that will calling the constructor
> inside a thread wrapper, only run the constructor inside that thread and
> then the thread will end, or will the entire object now run inside that
> thread from now on?  Am I being oversimplistic in my speculation?


If you want to force an object to, as you say, "run in one thread", you
would have to be able to make public every member function perform a context
switch to the desired thread upon entering the function and switch back upon
exiting.  You would have to protect all member variables and use Get/Set
functions for them that performed context switches as well.

Under Windows NT, if you send a message to a window created by a different
thread, that context switch is performed for you by the operating system.
Your process waits until the ::SendMessage() call completes.  Other than
using SendMessage(), I do not know how you would accomplish such an
operation.  And SendMessage requires a window to which the message will be
sent.  Thus, under NT, you would have to make your object create some kind
of hidden window in the context of the desired thread and then have every
member function do a ::SendMessage() to that window.

(There are variations -- e.g. SendMessageCallback(), PostMessage(), etc for
asynchronous function calls)

Such a design is possible, and maybe workable, but seems to defeat the
purpose of threads, doesn't it?  If one thread is just going to have to wait
for the special thread every function call, why have the special thread at
all?

And I haven't even considered OLE and accessing objects across process
boundaries, or thread-local storage.

(Again, I'm speaking pretty exclusively about the NT threading model here.
I've had enough VMS to last me a lifetime and know very little about Posix
threads.)

It seems your reason for wanting the entire object to run in its own thread
is to be true the OO paradigm, but I think that's perhaps too much of a good
thing.

Why not make your objects completely thread-safe instead?  Create some sort
of a Single-Writer / Multiple-Reader resource locking object for all objects
of the class.  Make each member function use this resource guard, acquiring
a read-lock if it's a const member function or write-lock if it is not
const.

There's nothing to prevent you from assigning specific threads to the
objects to do background work on them, but as long as all access to the
objects is through those safe member functions, they are completely thread
safe..

I mention this because this is how I designed a large radar project I just
finished working on.  I used completely thread-safe, reference counted
objects, read/write locks, and smart pointers in my design and the results
were far better than my most optimistic hopes.  A very fast workstation
program with many dynamic displays showing an enormous amount of continously
changing data stored in a SQL server database.

I've gone on way too long here so I'll end this without saying half of what
I want to say.  Hope this gives you a few ideas.
================================================================
 Q64: Can thread stacks be built in privately mapped memory?  

I've avoided any response to this long thread for a while because I'm not
sure I want to confuse the issue with facts. And, despite, the facts, I like
the idea of people learning to treat thread stacks "as if they might be"
private.

Nevertheless, at some time I thought it might be helpful to point out what
POSIX says about the matter... and I guess this is a good time.

POSIX very specifically disallows "non-shared" memory between threads.  That
is, it requires that the address space is associated with the PROCESS, not
with the individual THREADS. All threads share a single virtual address
space, and no memory address is private. Stacks, in particular, CANNOT be
set up with private memory. Although, for safe programming, you should
almost always pretend that it's private.

/---[ Dave Butenhof ]-----------------------[ butenhof@zko.dec.com ]---\
|  Digital Equipment Corporation         110 Spit Brook Rd ZKO2-3/Q18  |
|  603.881.2218, FAX 603.881.0120                Nashua NH 03062-2698  |
\-----------------[ Better Living Through Concurrency ]----------------/

================================================================
 Q65: Has anyone implemented a mutex with a timeout?  

Has anyone implemented a mutex locking function on top of Solaris or POSIX
threads with a timeout?  The problem I'm trying to solve is if a thread is
unable to obtain a mutex after a certain timeframe (say 30 seconds), then I
want the thread to terminate and return an error.  The Solaris and POSIX
API's only allow the user to check if a mutex can be obtained.

Of course! Check out the code for pthread_np_timed_mutex_t at
http://www.lambdacs.com/jan-97/examples.html

================================================================
 Q66: I think I need a FIFO mutex for my program...  

>There are VERY few cases where "lock ordering" is truly necessary. In
>general, when it may seem to be necessary, using a work queue to distribute
>the work across a pool of threads will be easier and more efficient. If
>you're convinced that you need lock ordering, rather than POSIX wakeup
>ordering, you have to code it yourself -- using, essentially, a work queue
>model where threads wishing to lock your "queued mutex" queue themselves in
>order. Use a condition variable and "next waiter" predicate to ensure proper
locking order. It's not that hard.

Right, and you can find a freely available implementation of essentially a
"FIFO Mutex" in ACE.  Take a look at

http://www.cs.wustl.edu/~schmidt/ACE_wrappers/ace/Token.h
http://www.cs.wustl.edu/~schmidt/ACE_wrappers/ace/Token.i
http://www.cs.wustl.edu/~schmidt/ACE_wrappers/ace/Token.cpp

Doug
--
Dr. Douglas C. Schmidt                  (schmidt@cs.wustl.edu)
Department of Computer Science, Washington University
St. Louis, MO 63130. Work #: (314) 935-4215; FAX #: (314) 935-7302
http://www.cs.wustl.edu/~schmidt/

You can also find an implementation of FIFO mutexes in the file
pthread_np.{h,c} at: http://www.lambdacs.com/jan-97/examples.html
================================================================
 Q67: Why my multi-threaded X11 app with LinuxThreads crashes?  
> Wolfram Gloger wrote:
> >
> > jenoll@cs.nmsu.edu (Jeff Noll) wrote:
> >
> > >       I'm making an X client that connects to a tcp socket. I'm using a
> > > thread to continually read from that socket connection and a text
> > > widget to sent to the socket. (an X telnet program that looks kind of
> > > like ncftp, seperate input/output windows). When i run this at school
> > > under solaris it seems to be fine, but when i take it home and try it
> > > under linus using linuxthreads 0.5 it crashes when i start entering
> > > into the text window.
> >
> > Crash as in `fatal' X errors ?  A while ago I had a similar experience
> > when trying to create a multi-threaded X11 app with LinuxThreads.  It
> > was quite easy to debug though: the LinuxThreads libpthread library
> > lets all threads get individual errno values (like they should), as
> > long as all sources are compiled with _REENTRANT defined.
> >
> > The X11 libs (at least in XFree86-3.2) are by default not compiled in
> > this way, unfortunately (note I'm not talking about multiple thread
> > support in X11), and they break when using LinuxThreads, e.g. because
> > Xlib relies on read() returning with errno==EAGAIN at times.  This is
> > a problem even when one restricts oneself to using X from a single
> > thread only.
> >
> > Once I recompiled all X11 libs with -D_REENTRANT (totally independent
> > of libpthread), everything works fine.  I could put those libs up for
> > ftp if you're interested to check it out.
> >
> > Regards,
> > Wolfram.
> 
================================================================
 Q68: How would we put a C++ object into a thread?  

> > > Been there. Done that. Let the constructor create a thread before
> > > returning to the caller (another object). But beware, your OS will
> > > propably start the thread by calling a function (specified by you)
> > > C-style. You want this function to be a member function in your class,
> > > which is ok as long as you make it static. The thread function will
> > > also need the this-pointer to your newly created object. What you want
> > > will look something like this (in NT):
> > >
> > > // Thread callback function.
> > > // NOTE: Need to be written in C or be a static member function
> > > // because of C style calling convention (no hidden this pointer)
> > > LPTHREAD_START_ROUTINE CThread::ThreadFunc(LPVOID inputparam)
> > > {
> > >    CThread *pseudo_this = (CThread *) inputparam;
> > >    ...
> > > }
> > >
> > > This function have access to all the members in the object through the
> > > pseudo this pointer. And all member functions called by this function
> > > will run in the same thread. You'll have to figure out how to
> > > communicate with the other objects in your system though. Be careful.
> > >
> > > --
> > > BaBu
> >
> > You can take this even a step further.  Add a pure virtual to your generic
> > CThread class like so:
> >
> > class CThread
> > {
> >       ...
> > protected:
> >     // I don't remember what Win32 expects as the return value, here,
> >     // but you can fix this up as you wish:
> >     virtual unsigned entryPoint() = 0;
> >       ...
> > };
> >
> > Then have the static ThreadFunc call it like so:
> >
> > // Thread callback function.
> > // NOTE: Need to be written in C or be a static member function
> > // because of C style calling convention (no hidden this pointer)
> > LPTHREAD_START_ROUTINE CThread::ThreadFunc(LPVOID inputparam)
> > {
> >    return ((CThread*) inputparam)->entryPoint();
> > }
> >
> > Now, to create a specific thread, derive from CThread, override entryPoint,
> > and you no longer have to mess around with a pseudo-this pointer, because
> > the real this pointer is available.
> >
> > One tricky issue:  make sure you differentiate between methods that the
> > thread itself will call, and methods that other threads (such as the one
> > that created the thread object) will call -- you will need to do thread
> > synchronization on class members that are shared data.
> >
> > Ian
> >
> > ___________________________________________________________________________
> > Ian Emmons                                       Work phone: (415) 372-3623
> > ian@persistence.com                              Work fax:   (415) 341-8432
> > Persistence Software, 1720 S. Amphlett Blvd. Suite 300, San Mateo, CA 94402
> 
================================================================
 Q69: How different are DEC threads and Pthreads?  
Mike.Lanni wrote:
> 
> Baard Bugge wrote:
> >
> > According th the thread-faq, DCE threads (as in HPUX 10.10) is an
> > older version of Posix 1003.1c threads (as in Solaris 2.5).
> >
> > Whats the differences? Is the two of them, fully or partly, source
> > code compatible?
> >
> > I want my multirhreaded code to be cross-compilable on at least the
> > two platforms mentioned above, without too many ifdefs. Can I?
> >
> > --
> > BaBu
> 
> Unfortunately, this is not black and white. If HPUX 10.10 is based on
> Draft 7 or higher, the Solaris and HP codes should be similar. However,
> if HP 10.10 is based on Draft 4, then there is quite a bit of work to be
> done. D4 became popular due to its usage with DCE. Assuming the worst,
> D4, here are some notes that I've put together based on some programming
> I've done. It is not complete by any means, but it should give you an
> idea of what you are up against.
> 
>  - signal handling is different
>  - return codes from pthreads api's are now the real error, vrs. -1
> and    errno
>  - possibly no support for the "non-portable" apis and symbolic
> constants
>  - non support for DCE exception handling
>  - Some of the pthread_attr_ apis have different types and arguments.
>  - Some of the scheduling apis have changed.
>  - Some thread specific api's have changed parameters.
> 
> Below are some mappings that at one time were valid...
> 
> #if defined(_D4_)
> #define PTHREAD_ONCE pthread_once_init
> #define PTHREAD_ATTR_DEFAULT pthread_attr_default
> #define PTHREAD_MUTEXATTR_DEFAULT pthread_mutexattr_default
> #define PTHREAD_CONDATTR_DEFAULT pthread_condattr_default
> #define INITROUTINE pthread_initroutine_t
> #define PTHREAD_ADDR_T pthread_addr_t
> #define START_RTN pthread_startroutine_t
> #define PTHREAD_YIELD pthread_yield
> #define PTHREAD_ATTR_DELETE pthread_attr_delete
> #define PTHREAD_ATTR_CREATE pthread_attr_create
> #define PTHREAD_MUTEXATTR_DELETE pthread_mutedattr_delete
> #define PTHREAD_MUTEXATTR_CREATE pthread_mutedattr_create
> #define PTHREAD_CONDATTR_DELETE pthread_condattr_delete
> #define PTHREAD_CONDATTR_CREATE pthread_condattr_create
> #define PTHREAD_KEYCREATE pthread_keycreate
> #define ATFORK atfork
> #define SIGPROCMASK sigprocmask
> #else
> #define PTHREAD_ONCE PTHREAD_ONCE_INIT
> #define PTHREAD_ATTR_DEFAULT NULL
> #define PTHREAD_MUTEXATTR_DEFAULT NULL
> #define PTHREAD_CONDATTR_DEFAULT NULL
> #define INITROUTINE void *
> #define PTHREAD_ADDR_T void *
> #define START_RTN void *
> #define PTHREAD_YIELD sched_yield
> #define PTHREAD_ATTR_DELETE pthread_attr_destroy
> #define PTHREAD_ATTR_CREATE pthread_attr_init
> #define PTHREAD_MUTEXATTR_DELETE pthread_mutedattr_destroy
> #define PTHREAD_MUTEXATTR_CREATE pthread_mutedattr_init
> #define PTHREAD_CONDATTR_DELETE pthread_condattr_destroy
> #define PTHREAD_CONDATTR_CREATE pthread_condattr_init
> #define PTHREAD_KEYCREATE pthread_key_create
> #define ATFORK pthread_atfork
> #define SIGPROCMASK pthread_sigmask
> #endif
> #if defined(_D4_)
>       rc = pthread_detach(&tid);
>       rc = pthread_exit(status);
>       rc = pthread_join(tid, &status);
>          pthread_setcancel(CANCEL_OFF);
>          pthread_setcancel(CANCEL_ON);
>     (void) pthread_setscheduler(pthread_self(),SCHED_FIFO,PRI_FIFO_MAX);
> #else
>       rc = pthread_detach(tid);
>       rc = pthread_exit(&status);
>       rc = pthread_join(tid, &status_p);
>          pthread_setcancelstate(PTHREAD_CANCEL_DISABLE,NULL);
>          pthread_setcancelstate(PTHREAD_CANCEL_ENABLE,NULL);
>     struct sched_param param;
>     param.sched_priority = 65535;
>     (void) pthread_setschedparam(pthread_self(),SCHED_FIFO,¶m);
> #endif /* _D4_ */
> 
> Hope this helps.
> 
> Mike L.
> --------------------------------------------------------------------
> Michael J. Lanni
> NCR                            email:  mike.lanni@columbiasc.ncr.com
> 3325 Platt Springs Road        phone:  803-939-2512
> West Columbia, SC 29170          fax:  803-939-7317
> http://www.columbiasc.ncr.com/home_pages/mlanni.html
================================================================
 Q70: How can I manipulate POSIX thread IDs?  
Steven G. Townsend wrote:
> 
> Jim Robinson wrote:
> >
> > In article <32DF9330.7D74@persistence.com>, Ian Emmons wrote:
> > >Robert Patrick wrote:
> > >>
> > >> Yes, you can copy one pthread_t to another.  The part you have to be
> > >> careful about is that in some implementations pthread_t is a struct
> > >> and in others it is not.  Therefore, setting two pthread_t's to be
> > >> equal by assignment will not be portable.  However, memcpy(a, b,
> > >> sizeof(pthread_t)) should always work.
> 
> As to the assignment issue, see Jim's comment below.
> As to the first point, assume for the moment that that a part of
> the structure is an array (status, pointers to currently allocated
> keys, whatever) if anything in the array can change, the "copy"
> will not be updated. Assume a status flag/bit which indicates
> whether the thread is runnable, looking at the copy could easily
> produce different results than the actual value of the "true"
> pthread_t.  This is just a bad thing to do.  Other problems
> can occur as well...
> What happens if both the original and the copy are passed to
> pthread_destroy?
> What happens if as we are doing the '=' or memcpy operation the
> thread is currently executing on a different processor (i.e.
> The contents of the pthread_t object would neeed to be protected
> by a mutex)?
> When it comes to copying pthread_t s...
>   Just say 'no'.
> 
> > >>
> > >> Just my two cents,
> > >> Robert
> > >
> > >Since I work in C++ exclusively, this isn't an issue for me, and so I never thought
> > >about that.  For C coders, you're right, of course.
> >
> > Structure assignment is defined in *ANSI* C. See page 127 of K&R, 2nd
> > edition. Since ANSI C has been standardized for quite some time now, it
> > should be a non-issue for C coders as well, no?
> 
> > --
> > Jim Robinson
> > robinson@mdd.comm.mot.com
================================================================
 Q71: I'd like a "write" that allowed a timeout value...  

Marc Peters wrote:
> What would be nice to have is a "write" that allowed a timeout value to be
> specified.  A la:
> 
>         write(fdSocket, bufferPtr, bufferLength, timeSpec);
> 
> If the write doesn't succeed withing the specified timeSpec, then errno
> should be set to ETIMEOUT or something.  Obviously, this would be quite handy
> in network code.
>
> Due to other circumstances beyond my control, the fdSocket cannot be placed
> in non-blocking mode.  Thus, the solution I'm left with is to start a POSIX
> timer, wait on the blocking write, and check for an errno of EINTR when it
> returns (if it timed out).
>
> I'm aware of the alternate technique of dedicating a thread to dispatching
> signal events.  This dedicated thread merely does a sigwaitinfo() and
> dispatches accordingly.  These technique, too, is offensive for such a simple
> requirement -- the "timed" write.

Why not just do these possibly long writes in separate threads? And, if
some "manager" decides they've gone too long, cancel the threads.

> I've ordered the POSIX 1003.1 standard to pursue this; however, it will be
> several days before it arrives.  Can anyone fill me in with some details of
> SIGEV_THREAD in the meantime?

SIGEV_THREAD creates a thread instead of raising a signal in the more
conventional manner. You get to specify the start routine (instead of
a signal catching function), and the attributes. The thread runs
anonymously (by default it's detached, and if you use an attributes
object with detachstate set to PTHREAD_CREATE_JOINABLE, the behavior
is unspecified).

The main advantage is that your "signal catching function" can lock
mutexes, signal condition variables, etc., instead of being restricted
to only the short list of async-signal safe functions.

In your case, a SIGEV_THREAD action would still constitute a "double
dispatch" for your signal. The code wouldn't look much different from
your current version.

Oh yeah, there's a major disadvantage, for you, in SIGEV_THREAD.
Solaris 2.5 doesn't implement SIGEV_THREAD. So you'd have to wait for
Solaris 2.6. (Just to be fair, I'll also point out that Digital UNIX 4.0
didn't do SIGEV_THREAD, either -- it is a minor and relatively obscure
function, and we all had more important things to worry about. We also
caught up to it later, and we'll be supporting SIGEV_THREAD in Digital
UNIX 4.0D [or at least, that's what we're calling it now, though these
things are always subject to change for various reasons].)

/---[ Dave Butenhof ]-----------------------[ butenhof@zko.dec.com ]---\
| Digital Equipment Corporation           110 Spit Brook Rd ZKO2-3/Q18 |
| 603.881.2218, FAX 603.881.0120                  Nashua NH 03062-2698 |
\-----------------[ Better Living Through Concurrency ]----------------/

================================================================
 Q72: I couldn't get threads to work with glibc-2.0.  

>I finally got this to compile cleanly, but it stalls (sigsuspend) somewhere
>in pthread_create()...

	If you are using glibc-2.0, you should upgrade to glibc-2.0.1. 
There is a bug in 2.0 that makes all thread creation fails. 

	     Yann Doussot  
================================================================
 Q73: Can I do dead-owner-process recovery with POSIX mutexes?  

el Gringo wrote:
> 
> Hi.
> 
> I am trying to create a mutxe on a program that has to work on NT 4.0 and AIX.
> For NT, I use CreateMutexes...etc, and in this case, if the process owning the
> mutex crashes, the system releases the mutex and returns WAIT_ABONDONED to the
> thread that is waiting for the mutex to be released. And if the mutex is
> opened several times by the same thread, the call succeses and the mutex count
> is incremeneted.
> 
> What I don't know is if the pthread mutexes do the same thing when a thread or
> process owning the mutexe crashes...or when the Pthread_mutex_lock() is called
> several times by the same one. Could someone provide me with a doc or web site
> so I can find those answers ? Thanks. Riad

Riad,

  Ah...

  You got a *big* set of problems to deal with.  The simple answer is that
POSIX mutexes don't do that, but that you can create that kind of behavior
if you want to.

  The problems I refer to are those surrounding what you do when the owner process
crashes.  There is a massive amount of work to be done to ensure that you don't
have corrupted data after you get WAIT_ABANDONED.  Unless you've already taken
care of this, you've got a h*** of a lot of work to do.

  So...  the best answer is go find an expert in this area (or spend a month
becoming one) and hash out the issues.  Building the mutex will be the easiest
part of it.

  Good luck.

-Bil
================================================================
 Q74: Will IRIX distribute threads immediately to CPUs?  

Michel Lesoinne   wrote:
>The first question concerns new and delete under C++ as well as malloc.
>Are these functions thread-safe? 
------------------------------------------------------------------------
Yes, these calls are thread-safe.

>The second question has to do with multi-CPU machines. I have noticed
>that POSIX thread do not get shipped to other CPU's immediately after
>being started. For example if you bring up gr_osview and watch the CPU
>usage as you start 4 paralll threads, it takes approximately 1 second
>for the 4 threads to run in parallel rather than on the same CPU. Worse,
------------------------------------------------------------------------
The current pthread implementation only creates additional sprocs as it
deems necessary given the application activity.  Interaction between
pthreads which may require context switching can lower the requirement
while CPU-bound threads will raise it.  There can be a short delay
"ramping-up" before the ideal number of sprocs are active.  The kernel
is responsible for scheduling the sprocs on CPUs.  Thus you may be
seeing 2 effects (though 1 second seems a little long to me).

>Is there a way to force IRIX to distribute the threads immediately?
------------------------------------------------------------------------
Currently there is no way to force this behaviour though we expect to
add tuning interfaces in the future.  You may try experimenting by setting
the environment variable PT_ITC as a hint to the library that your app is
CPU bound.

	sincerely,
		..jph...
================================================================
 Q75: IRIX pthreads won't use both CPUs?  

Dirk Bartz   wrote:
>I've written a parallel program using pthreads on two processors
>which consists of two parallel stages.
>In first stage reads jobs from a queue (protected by a mutex) and
>should process them parallelly (which it doesn't); the second
>stage works fine.
>
>Now, some debugger sessions of the first stage show that
>both pthreads are started, but the first one is being blocked
>most of the time. The cvd says:
>   0	_procblk() ["procblk.s":15, 0x0fab4d74]
>   1	_blockproc() ["blockproc.c":23, 0x0fab56e0]
>   2	vp_idle() ["vp.c":1702, 0x07fe61e8]
>
>It seems that the first pthread is only sharing one processor
>with the second thread. It is *not* blocked at the mutex!
>
>Does anyone has a clue what happend?
------------------------------------------------------------------------
Hi,
	First of all that curious backtrace is from one of the underlying
sprocs on which the pthreads execute.  As you can see it is currently idle
and blocked in the kernel waiting for the library to activate it when more
work (a pthread) is ready to run.  If you use cvd showthread all command
it will show you the pthread state which should in your case be MUTEX-WAIT
for the pthread of interest.  If you then backtrace that pthread you should
see it blocked in the mutex locking code.

A second point to note is that the pthreads library attempts to use an
appropriate number of sprocs for its scheduling.  If your application creates
2 CPU-bound threads then on an MP machine 2 sprocs will be created to run
the threads.  On a UP only one sproc will be created and will switch between
the two pthreads.  On an MP where the threads are not CPU-bound the problem
is more complex; when 2 pthreads are tightly synchronised then a single
sproc may be a better choice - this may be what you are seeing.

I hope the above explains what you are seeing.

	sincerely,
		..jph....
================================================================
 Q76: Are there thread mutexes, LWP mutexes *and* kernel mutexes?  

> In a typical "two level" scheduling scheme, say solaris,
> synchronization primitives used at the thread level (POSIX or solaris)
> are provided by the user level scheduler library.  At the LWP level,
> are there any synchronization primitives, and if so, where would one
> use those as opposed to using the user level library primitives?
> Ofcourse, there would be some synchronization primitives for the
> kernel use.  Does it mean that there are 3 distinct set of primitives
> (user level, LWP level and kernel level)?  Can anyone throw some light
> on the LWP level primtives (if any) and point out where these would be
> useful?

  You may remember that scene in the Wizard of Oz, where Toto runs away
in panic at the sight of the Powerful Oz.  He discovers a little man 
running the machinery behind a curtin.  The catch-line was "Pay no attention
to that man behind the curtin."

  Same thing here.  You call pthread_mutex_lock() and it does whatever it
needs to so that things work.  End of story.

  But if you *really* want to peek...  If the mutex is locked, then the
thread knows that it needs to go to sleep, and it calls one set of routines
if it's an unbound thread, another if it's bound.  (If you hack around inside
the library code, you'll be able to see the guts of the thing, and you'll
find calls to things like _lwp_mutex_lock().  You will NEVER call those!)

  Now, as for kernel hacking, that's a different picture.  If you are going
to go into the kernel and write a new device driver or fix the virtual 
memory system, you'll be working with a different interface.  It's similar
to pthreads, but unique to each kernel.  The older OSs didn't even HAVE a
threads package!
================================================================
 Q77: Does anyone know of a MT-safe alternative to setjmp and longjmp?  
================================================================
>      I am taking an operating systems class; therefore, my
>    question will sound pretty trivial.  Basically, I am
>    trying to create thread_create, thread_yield, and thread_exit
>    functions.  Basically, I have two files.  They compile fine and
>    everything but whenever I try to run the program I get the error:
>    "longjmp or siglongjmp function used outside of saved context
>     abort process"
>    All I know is that we are running this on an alpha machine at
>    school [...]
> 
>    Anyway, I just want to know if anyone has ever tried to do a longjmp from a
>    jmp_buf that was not the same as that used in setjmp.
> 
> The runtime environment provided with some operating systems (e.g.,
> Ultrix or whatever DEC `Unix' is called these days) performs an
> explicit check that the destination stack frame is an ancestor of
> the current one.  On these systems you cannot use setjmp/longjmp
> (as supplied) to implement threads.
> 
> On systems whose longjmp is trusting, setjmp/longjmp is a very common
> way of building user-space threading libraries.  This particular wheel
> has been reinvented many times.
> 
> If you know the layout of a jmp_buf, you *can* use setjmp but you will
> have to implement a compatible longjmp yourself in order to change the
> processor context to that of the next task.  If you have a
> disassembler you might be able to reverse engineer a copy of longjmp
> with the check disabled.
> 
> *I* would consider this outside the scope of such an exercise but your
> professor may disagree.
> 
> Steve
> --
> Stephen Crane, Dept of Computing, Imperial College of Science, Technology and
> Medicine, 180 Queen's Gate, London sw7 2bz, UK:jsc@{doc.ic.ac.uk, icdoc.uucp}
> Unix(tm): A great place to live, but a terrible place to visit.
================================================================
 Q78: How do I get more information inside a signal handler?   
Mark Lindner wrote:
> 
> I'm writing a multithreaded daemon that supports dynamic runtime loading
> of modules (.so files). I want it to be able to recover from signals such
> as SIGSEGV and SIGFPE that are generated by faulty module code. If a given
> module causes a fault, I want the daemon to unload that module so that
> it's not called again.
> 
> My problem is that once a signal is delivered, I don't know which worker
> thread it came from, and hence I have no idea which module is faulty. The
> O'Reilly pthreads book conveniently skirts this issue. I poked around on
> the system and found the getcontext() call; I tried saving the context for
> each worker thread, and then using the ucontext_t structure passed as the
> 3rd argument to the signal handler registered by sigaction(), but
> unforunately I can't find anything that matches...the contexts don't even
> appear to be the same.
> 
> Since the behavior of pthreads calls is undefined within a signal handler,
> I can't use pthread_self() to figure out which thread it is either.
> 
> All examples I've seen to date assume that either:
> 
> a) only one thread can generate a given signal
> 
> or
> 
> b) two or more threads can generate a given signal, but the signal handler
> does the same thing regardless of which thread generated it.
> 
> My situation doesen't fall into either of these categories.
> 
> Any help would be appreciated.
> 
> --
> 
> Cheers!
> Mark
> 
> ------------------------------------------------------------------------------
> markl@love.geology.yale.edu       | http://love.geology.yale.edu/~markl/
> ------------------------------------------------------------------------------
>                    I looked up
>                    As if somehow I would grasp the heavens
>                    The universe
>                    Worlds beyond number...
> ------------------------------------------------------------------------------
================================================================
 Q79: Is there a test suite for Pthreads?  


Re: COMMERCIAL: Pthreads Test Suite Available
 
The Open Group VSTH test suite for Threads implementations of
POSIX 1003.1c-1995 and  the X/Open System Interfaces (XSH5) Aspen
threads extensions is now generally available.
For further information on the test suite see
http://www.rdg.opengroup.org/testing/testsuites/vsthover.htm
For information on the Aspen threads extensions see
http://www.rdg.opengroup.org/unix/version2/whatsnew/threads.html

> Andrew Josey, Email: ajosey@rdg.opengroup.org
> #include 
================================================================
 Q80:  Flushing the Store Buffer vs. Compare and Swap  

Just looking at the CAS and InterLockedXXX instructions... 

  "Hey!" says I to myself, "Nobody's minding the store buffer!"
A couple of people have shown some examples of using InterLockedXXX
in Win32, but they never address memory coherency!

  So, if they implement a mutex with InterLockedExchange:

lock(int *lock)
{while (InterLockedExchange(lock, 1) == 1) sleep();} 

unlock(int *lock)
{*lock = 0;}

  at unlock time, some changed data might not be written out to
main memory.  Hence we need this:

unlock(int *lock)
{
 FlushStoreBuffer();
 *lock = 0;
}

  Or is there something about x86 that I don't know about here?
================================================================
 Q81: How many threads CAN a POSIX process have?  

Dave Butenhof wrote:
> 
> Bryan O'Sullivan wrote:
> >
> > r> _POSIX_THREAD_THREADS_MAX that claims to be the maximum threads per
> > r> process.
> >
> > As I recall, this is a minimum requirement.  Solaris certainly
> > supports far more than 64 threads in a single process, and I'm sure
> > that Irix does, too.
> 
> POSIX specifies two compile-time constants, in , for each
> runtime limit. One is the MINIMUM value of that MAXIMUM which is
> required to conform to the standard. _POSIX_THREAD_THREADS_MAX must be
> defined to 64 on all conforming implementations, and all conforming
> implementations must not arbitrarily prevent you from creating at least
> that many threads.
> 
> The symbol PTHREAD_THREADS_MAX may ALSO be defined, to the true limit
> allowed by the system, IF (and only if) that limit is fixed and can be
> predicted at compile time. (The value of PTHREAD_THREADS_MAX must be at
> least 64, of course.) I don't know of any systems that define this
> symbol, however, because we don't implement any fixed limit on the
> number of threads. The limit is dynamic, and dictated purely by a wide
> range of resource constraints within the system. In practice, the only
> way to predict how many threads you can create in any particular
> situation is to bring a program into precisely that situation and count
> how many threads it can create. Remember that the "situation" includes
> the total size of your program text and data, any additional dynamic
> memory used by the process (including all shared libraries), the virtual
> memory and swapfile limits of the current system, and, in some cases,
> the state of all other processes on the system.
> 
> In short, the concept of "a limit" is a fiction. There's no such thing,
> without knowing the complete state of the system -- rarely practical in
> real life.
> 
> Oh, and by the way, there's no guarantee (in POSIX or anywhere else)
> that you can create even 64 threads. That just means that the system
> cannot arbitrarily prevent you from creating that many. If you use up
> enough virtual memory, you may be unable to create even one thread.
> That's life.
> 
> As Bryan said, you can normally rely on being able to create hundreds,
> and usually thousands, of threads on any of the current 2-level
> scheduling POSIX threads implementations. Kernel-level implementations
> are typically more limited due to kernel quotas on the number of kernel
> thread slots available for the system and often for each user.
================================================================
 Q82: Can Pthreads wait for combinations of conditions?  

> Is there any way in Pthreads to wait for boolean combinations of conditions
> (i.e. wait for any one of a set of conditions or wait until all of a set of
> conditions have occurred). I'm looking for a feature similar to the VMS
> Wait for logical OR of event flags or the OS/2 multiplexed semaphores.

  You mean something like this:


void *consumer(void *arg)
{request_t *request;

 while(1)
   {pthread_mutex_lock(&requests_lock);
    while ((length == 0) && (!stop)) 	<--  While both are true, sleep
      pthread_cond_wait(&requests_consumer, &requests_lock);
    if (stop) break;
    request = remove_request();
    length--;
    pthread_mutex_unlock(&requests_lock);
    pthread_cond_signal(&requests_producer);
    process_request(request);
  }
 pthread_mutex_unlock(&requests_lock);
 sem_post(&barrier);
 pthread_exit(NULL);
}


Or perhaps:

    while ( ((length == 0) && (!stop))	||
	    (age_of(granny) > 100) ||
	    (no_data_on_socket(fd) && still_alive(client)) ||
	    (frozen_over(hell))  )
      pthread_cond_wait(&requests_consumer, &requests_lock);


  Nope.  Can't be done  :-)


  Now if you're thinking about something that involves blocking, it may be
a bit trickier.  In OS/2 or Win32 you might think in terms of:

  WaitForMultipleObjects(Mutex1 and Mutex2)

you'll have to do a bit extra.  Perhaps you'll have two different threads 
blocking on the two mutexes:

	Thread 1
 pthread_mutex_lock(Mutex1);
 M1=TRUE;
 pthread_cond_signal(&requests_consumer);


	Thread 2
 pthread_mutex_lock(Mutex2)
 M2=TRUE;
 pthread_cond_signal(&requests_consumer);

	Thread 3
 while (!M1 || !M2)
   pthread_cond_wait(&requests_consumer, &requests_lock);



  I think this looks sort of ugly.  More likely you'll find a better way 
to structure your code.



================================================================
 Q83: Shouldn't pthread_mutex_trylock() work in shared memory even if it's NOT PTHREAD_PROCESS_SHARED?  

Curt,

  I infer your're trying to get around the lack of shared memory SVs
in some of the libraries by only using trylock?  I can't say that I
approve, but it ought to work...

  In the code example below I hacked up an example which does seem to
do the job.  I can't tell you what you were seeing in your tests.
Hmmm...  Just because this particular hack works on one OS does not
mean that it will necessarily work on another.  (Let's say I wouldn't
stake MY job on it!) 

  What about using shared semaphores?  Maybe SysV semaphores?

-Bil

> HI,
> 
> I'm having a problem with pthread_mutex_unlock () on Solaris 2.5 for a
> pthread_mutex_t inited in a shared memory structure between 2 tasks.
> 
> I get pthread_mutex_trylock (lockp) to return zero, and both tasks
> agree the mutex is locked.
> 
> When the owning task calls pthread_mutex_unlock (lockp), it returns
> zero, but the other task's pthread_mutex_trylock (lockp) still believes
> the mutex is locked??
> 
> FAQ location or help?  Thanks.
> 
> Heres how I initted the pthread_mutex_t struct:
> 
> In a shared memory struct:
> 
> pthread_mutex_t         mutex_lock =  PTHREAD_MUTEX_INITIALIZER;
> 
> Then either task may call:
> 
> pthread_mutex_trylock (&mutex_lock)
> ...work...
> pthread_mutex_unlock (&mutex_lock)
> 
> I've had little luck with  pthread_mutexattr_setpshared () to init for
> the "shared" scenario (core dumped) and especially that this is a Sun'ism
> that doesn't exist in DEC Unix 4.0b, which is another requirement, that
> the solution be portable to all/most Unix'es with threads.
> 
> Thanks.
> 
> Curt Smith
> curts@compgen.com



       sleep(4-i/2);	/* wait a second to make it more interesting*/
      if (!err) {pthread_mutex_unlock(&buf->lock2);
		  printf("Unlocked by parent\n");
		}
   }

  printf("Parent PID(%d): exiting...\n", getpid());
  exit(0);
}


main(int argc, char *argv[])
{int fd;
 pthread_mutexattr_t mutex_attr;

 /* open a file to use in a memory mapping */
 fd = open("/dev/zero", O_RDWR);

 /* Create a shared memory map with the open file for the data 
    structure which will be shared between processes. */

 buf=(buf_t *)mmap(NULL, sizeof(buf_t), PROT_READ|PROT_WRITE,
		   MAP_SHARED, fd, 0);

 /* Initialize the counter and SVs.  PTHREAD_PROCESS_SHARED makes
    them visible to both processes. */


 pthread_mutexattr_init(&mutex_attr);
/* pthread_mutexattr_setpshared(&mutex_attr, PTHREAD_PROCESS_SHARED); */

 pthread_mutex_init(&buf->lock2, &mutex_attr);

 if (fork() == 0)
   my_child();
 else
   my_parent();
}
================================================================
 Q84: What about having a NULL thread ID?  

This is part of an on-going discussion.  The POSIX committe decided not to
do this.  It is, of course, possible to implement non-portable versions
yourself.  You would have to have a DIFFERENT version for different OSs.
-1L works fine for Solaris 2.5, and IRIX 6.2, but not HP-UX 10.30, which
requires (I think!) {-1, -1, -1}.  BE CAREFUL!!!

Now for the discussion...

Ben Self wrote:
> 
> Ian Emmons wrote:
> >
> > So why not support a portable "null" value?  This
> > could be done via a macro that can be used to initialize a pthread_t (just
> > like the macro that initializes a mutex), or it could be done via a couple
> > of functions to set and test for the null value.  Or, the POSIX folks could
> > do as they are used to doing, and make us code around their ommissions with
> > YAC (Yet Another Class).
> >
> > Ian
> >
> As I stated before I think that it is a very natural thing to want to
> do.  In my experience POSIX's omissions are usually more interesting
> than simple oversights.  Often an existing industry code base stands in
> the way or it was deemed too 'trivial' a matter of user code to merrit
> imposing any restriction on implimentation.  in any case, for all
> intents and purposes, it is a done deal.
> 
> As Dave Butenhof candidly exposes a few posts down:
> 
> "due to
> overwhelming agreement that it was a bad (and unnecessary) complication.
> The Aspen committee added suspend/resume to UNIX98 -- but the functions
> were later removed with no significant objections.
> 
> There simply is no significant industry concensus supporting these
> functions. (And that for many very good technical reasons as well as
> some silly political reasons."
> 
> that is exactly how POSIX works.  gotta love 'em
> 
> --Ben
================================================================
 Q85: Explain Traps under Solaris  

Jim Moore - Senior Engineer, SunSoft wrote:
Email : Jim.Moore@UK.Sun.COM               |   DNRC: The Day Cometh
SunSoft Technical Support (Europe)         |   "adb is your friend"

    ---------------------------------------------------------------

                        SPARC traps under SunOS (Solaris)
                        -=-=-=-=-=-=-=-=-=-=-=-
                By: Jim Moore, SunSoft, Sun Microsystems Inc
                      Email: Jim.Moore@UK.Sun.COM

        CONTENTS

        1       Introduction
        1.1     Who should read this document?

        2       What is a trap?
        2.1     How traps are caused
        2.1.1   Precise Traps
        2.1.2   Deferred Traps
        2.1.3   Disrupt/Interrupt Traps
        2.2     How traps are dispatched to the kernel
        2.2.1   SPARC v7/v8
        2.2.2   SPARC v9
        2.2.2.1 Processor states, normal and special traps
        2.2.2.2 Normal Trap (Processor in Execute State)
        2.2.2.3 Special Trap (Processor in RED State)

        3       Traps - How SunOS Handles Them
        3.1     Generic Trap Handling
        3.2     Register Windows
        3.3     Interrupts

                        -=-=-=-=-=-=-=-=-=-=-=-=-

1       INTRODUCTION

        This document describes what traps are, how they work and how they
        are handled by the SunOS kernel.   We will look at traps for SPARC
        versions v7, v8 and v9 (v7 and v8 traps are essentially identical).

        In places, we will have to differentiate between the v8 and v9
        quite extensively as there are significant differences between the
        two.

        I assume that the readers are familiar with SPARC registers but I
        will give some expansion on the more obscure ones as I go :-)

        Finally, I have made every effort to make this accurate as well as
        informative but at the same time without too lengthy descriptions.
        Even so, in parts it may be heavy going and plain ascii doesn't
        leave much scope for clear diagrams.  Feel free to post or email
        questions and comments!

1.1     Who should read this document?

        Anyone who wants to know more about traps in detail  on  the SPARC
        architecture.  I strongly recommend that you refer to one of these
        two books for more information:

        The SPARC Architecture Manual, Version 8        ISBN 0-13-099227-5
        The SPARC Architecture Manual, Version 9        ISBN 0-13-825001-4

        as they contain more detail on some of these topics.

2       WHAT IS A TRAP?

        The design of SPARC as a RISC processor means that a lot of the
        functionality that is normally controlled by complex instructions
        has to be done by supervisor (kernel) software.  Examples of these
        could be memory exception handling or interrupt handling.  When
        a situation arises that requires special handling in this way,
        a trap occurs to cause the situation to be handled.  We'll look
        at this mechanism in more detail in this section.

2.1     How Traps are Caused

        Traps can be generated for a number of reasons and you can see
        a list of traps under /usr/include/sys/v7/machtrap.h or under
        /usr/include/sys/v9/machtrap.h for SPARC v7/v8 and v9 respectively.

        A trap can be caused either by an exception brought about by the
        execution of an instruction or by the occurrence of some external
        interrupt request not directly related to the instruction.  When
        the IU (Integer Unit, the part of the CPU that contains the
        general purpose registers, does integer math and executes the
        instructions) is about to execute an instruction it first checks
        to see if there are any exception or interrupt conditions pending
        and, if so, it selects the highest priority one and causes a trap.

        Traps are also used to signal hardware faults and malfunctions,
        for example a level15 asynchronous memory fault.  In some fatal
        conditions execution cannot continue and the machine will halt
        or the supervisor software will handle the trap by panicing.

        Next, we'll take a generic look at the different trap categories
        but we'll go into the version specific details later on.

2.1.1   Precise Traps

        A precise trap is brought about by an exception directly caused by
        the executing instruction.  This trap occurs before there is any
        tangible change in the program state of the program that contained
        the trapped instruction.

2.1.2   Deferred Traps

        A deferred trap is similar to a precise trap but in this case the
        program-visible state may have changed by the time the trap occurs.
        Such a trap may in theory occur one or more instructions after the
        trap inducing instruction has executed but it must occur before
        any subsequent instruction attempts to use any modified register
        or resource that the trap inducing instruction used.

        Did you get that?  Hmm.  Okay, here's an example.  Imagine that
        a floating point operation is being executed.  This does not
        happen synchronously with IU instructions and so it is possible
        that a floating point exception could occur as a deferred trap.

2.1.3   Disrupt/Interrupt Traps

        An interrupt trap, as you have probably guessed, is basically the
        assertion of an interrupt, either generated externally (from
        hardware) or internally (via software).  The delivery of interrupts
        is controlled by the PIL (Processor Interrupt Level) field of the
        PSR (Processor State Register), which specifies the minimum
        interrupt level to allow, and also by the mask of asserted bits in
        the IE (Interrupt Enable register...architecture specific).

        Under SPARC v9, we have a concept of a disrupt trap.  This is very
        similar to a deferred trap in that it could be related to an earlier
        instruction but in this case the trap is an unrecoverable error.

2.2     How Traps are Dispatched to the Kernel

        In this section we will look at the flow of execution into the
        kernel when a trap occurs.  This is different for SPARC v7/v8
        and v9 so we will split this section into two.

2.2.1   SPARC v7/v8

        When a trap occurs, the flow of execution jumps to an address which
        is calculated from the TBR (Trap Base Register) and the Trap Type,
        hereafter referred to as TT.  The sequence is as follows:

        1.  An exception/interrupt has been detected as pending by the IU

        2.  The IU multiplies the TT by 16 (TT << 0x4) as there are 4
            instructions per trap table entry.

        3.  The IU loads the address of the trap table (from the TBR) and
            adds the offset calculated in (2).

        4.  The CWP (Current Window Pointer) is decremented, so that we are
            in a new register window.

        5.  The trapped instruction (%pc) and the next instruction to be
            executed (%npc) are written into local registers %l1 and %l2

        6.  Traps are disabled and the current processor mode is set to
            "supervisor".  This is done by setting the ET bit to zero and
            the supervisor mode bit to one in the PSR (refer to the PSR
            description in /usr/include/v7/sys/psr.h).

        7.  Execution resumes at [TBR + (TT<<4)], as calculated in (3)

        Part of the SunOS kernel code is a trap table, which contains 255
        4-instruction entries, each entry corresponding to a trap type from
        0 to 0xff.  This structure is defined by the SPARC implementation.

        Each trap table entry basically contains a branch to a trap handling
        routine and may also load the PSR into a local register for use
        later.  Here's an example of a trap table entry:

                sethi   %hi(trap_handler), %l3          ! Load trap handler
                jmp     [%l3 + %lo(trap_handler)]       ! address and jump
                mov     %psr, %l0                       ! Delay: load %psr
                nop
 
2.2.2   SPARC v9

        The SPARC v9 case is quite different from the v7/v8 case mainly
        due to the concept of processor states and trap nesting.

        We still use a trap table concept under v9 but the destination
        address for the transfer of execution is calculated differently.
        Also, trap table entries for v9 are 8 instructions in size,
        except for spill/fill traps, in which case the entries are 32
        instructions in size.  Also, in a special state called the RED
        state (more on that later) we actually use a different trap table!

        Pretty different, huh?

        The trap table is divided into three parts.  The first half of the
        table is used for machine generated traps.  The next quarter is
        reserved for software initiated traps and the final quarter is
        reserved for future use.  The displacement into the trap table is
        defined by Trap Level (TL) and the Trap Type (TT) together.

        Let's take a look at this in some more detail.  I strongly advise
        that you obtain a copy of the version 9 SPARC architecture manual
        if you want to follow this in detail.

        When a trap occurs, the action taken depends on the TT, the current
        level of trap nesting (contained in the TL) and the processor state
        at that time.  Let's look at processor states and what we mean by
        normal and special traps so that the rest of this section has more
        chance of making sense!

2.2.2.1 Processor States, Normal and Special Traps

        The SPARC v9 processor is always in one of three states and these
        are:

        1.  Execute state.  This is the normal execution state.

        2.  RED state.  RED = Reset, Error and Debug.  This is a state that
            is reserved for handling traps when we are at the penultimate
            level of trap nesting or for handling hardware or software
            interrupts.

        3.  Error state.  This is a state that is entered when we have a
            trap occur at a point in time when we are at our maximum level
            of trap nesting (MAXTL) or an unrecoverable fatal error has
            occurred.

        Normal traps are traps that are processed when we are in the nice
        cosy execute state.  If we trap in RED state, then this is a special
        trap.  There is an implementation dependent address called RSTVaddr
        which contains the vector to the RED state trap table.  This vector
        could be set to overlay the same one in the TBR.  For a given trap
        in RED state we vector as follows:

        TT      Vector          Reason

        0       RSTVaddr|0x0    SPARC v8 style reset
        1       RSTVaddr|0x20   Power On Reset (POR)
        2       RSTVaddr|0x40   Watchdog Reset (WDR)
        3       RSTVaddr|0x60   Externally Initiated Reset (XIR)
        4       RSTVaddr|0x80   Software Initiated Reset (SIR)
        Others  RSTVaddr|0xa0   All other traps in RED state

        A fatal exception that causes us to drop into error state
        will cause the processor to note the exception and either halt,
        reset or watchdog reset.  After the reset, the processor enters
        RED state with a TL appropriate to the type of reset (usually
        maximum).  Also, the TT is set to the value of the original trap
        that caused the reset and NOT the TT value for the reset itself
        (ie. WDR - Watchdog reset or XIR - Externally Indicated Reset).

        Now that we have a concept of the different traps and processor
        states, let's look at the sequence of execution when a trap occurs
        to deliver the trap to the supervisor (kernel).

2.2.2.2 Normal Trap (Processor in Execute State)

        1.  If TL = MAXTL-1, the processor enters RED state (Goto 2.2.2.3).

        2.  TL = TL + 1

        3.  Processor state, %pc, %npc, CWP, CCR (Condition Code Register),
            TT and ASI (Address Space Identifier register) are saved.

        4.  The PSTATE (Processor State) register is updated as follows:

                a) RED field set to zero
                b) AM (Address Masking) disabled
                c) PRIV (Privileged Mode) enabled
                d) IE cleared, disabling interrupts
                e) AG set (Alternate Global Registers enabled)
                f) Endian mode set for traps (TLE)

            Refer to the architecture manual for a description of PSTATE

        5.  If TT is a register window trap, CWP is set to point to the
            register window to be accessed by the trap handler code.
            Possibilities are:

                a) TT = 0x24 (Clean Window), CWP = CWP + 1
                b) TT <= 0x80 AND TT <= 0xbf (Window Spill),
                   CWP = CWP + CANSAVE + 2.  CANSAVE is a register that
                   contains the number of register windows following the
                   CWP that are NOT in use.
                c) TT <= 0xc0 AND TT <= 0xff (Window fill),
                   CWP = CWP - 1

            For non-register window traps, CWP remains unchanged.

        6.  Control is transferred to the trap table at an address calculated
            as follows:

                New %pc =  TBA | (TL>0 ? 1: 0) | TL
                New %npc = TBA | (TL>0 ? 1: 0) | TL | 0x4

            Remember, TBA = Traptable Base Address, similar to the TBR in v8
            Execution then resumes at the new %pc and %npc

2.2.2.3 Special Trap (Processor in RED State)

        1.  TL = MAXTL

        2.  The existing state is preserved as in 2.2.2.2, step 3.

        3.  The PSTATE is modified as per 2.2.2.2, step 4 except that the
            RED field is asserted.

        4.  If TT is a register window trap, CWP processing occurs as in
            2.2.2.2, step 5.

        5.  Implementation specific state changed may occur.  For example,
            the MMU may be disabled.

        6.  Control is transferred to the RED state trap table subject to
            the trap type.  Look back to 2.2.2.1 for the RSTVaddr
            information to see how this vector is made.

        This may seem rather complicated but once you have the picture
        built clearly it will all fall into place.  Post or email if you
        need clarification.

3       TRAPS - HOW SUNOS HANDLES THEM

        In this section we will look at how SunOS handles traps and look
        at some of the alternatives which were available.  Despite all the
        differences between SPARC v8 and v9 traps I'll do a fairly generic
        description here as it really isn't necessary to describe in detail
        what SunOS does for v9 traps as you can see from the previous
        section what the differences in trap processing are.  Suffice to
        say that the SunOS kernel adheres to those rules.  Instead, we'll
        concentrate on the principles used by the kernel when handling
        various traps.

3.1     Generic Trap Handling

        We'll look at some specifics in a moment but first we'll cover the
        generic trap handling algorithm.

        When traps are handled, the typical procedure is as follows:

        1.  Check CWP.  If we need to handle the trap by jumping to
            'C' (which would use save and restore instructions between
            function calls) then we must make sure we won't have cause
            an overflow when we dive into 'C'.  If we do detect that
            this would be a problem we do the overflow processing now.

        2.  Is this an interrupt?  If so, jump to the interrupt handler.
            Refer to section 3.3 on interrupts.

        3.  Enable traps and dive into the standard trap handler.  We
            enable traps so that we can catch any exceptions brought
            about by handling *this* trap without causing a watchdog reset.

        4.  On return from the trap handler, we check the CWP with the
            CWP we came in with at the start to see if we have to undo
            the overflow processing we might have done before, so that
            we don't get an underflow when we return to the trapped
            instruction (or worse, execution continues in the WRONG window).

        5.  Before we actually return to the trapped instruction, we check
            to see if kprunrun is asserted (ie. a higher priority lightweight
            process is waiting to run).  If so, we allow preemption to
            occur.

        Traps are used by SunOS for system calls as well as for machine
        generated exceptions.  The parameters to the system call are
        placed in the output registers, the number of the system call
        required (see /usr/include/sys/syscall.h) is placed in %g1 and
        then it executes a "ta 0x8" instruction.  This appears in the kernel
        as a trap with TT = 0x88 and the system trap handler determines
        this to be a system call and calls the relevant function as per
        the system call number in %g1.
 
        Occasionally, a process will attempt to execute from a page of
        VM that is not mapped in (ie. it is marked invalid in the MMU)
        and this will cause a text fault trap.  The kernel will then
        attempt to map in the required text page and resume execution.
        However, if the process does not have the correct permissions
        or the mapping cannot be satisfied then the kernel will mark
        a pending SIGSEGV segmentation violation against that process
        and then resume execution of the process.  A similar scenario
        applies to data faults; a process attempts to read or write to
        an address in a page marked invalid in the MMU and the kernel
        will attempt to map in the corresponding page for this address
        if possible (ie. maybe the page has been swapped out or this
        is the first attempt to read from that page and so we demand-page
        it in).  I'll explain all this in detail in another text on
        process address spaces, paging and swapping which I plan to do
        as soon as I get time.

        A "bad trap" is simply a trap that cannot be handled (or isn't
        supported).  Usually under SunOS a bad trap has a type of 9 or
        2, for data or text fault respectively (maybe 7 for alignment in
        some cases).

3.2     Register Windows
================================================================
 Q86: Is there anything similar to posix conditions variables in Win32 API ?  

Ian Emmons wrote:
> 
> Dave Butenhof wrote:
> >
> > kumari wrote:
> > >
> > > Is there anything similar to posix conditions variables in Win32 API ?
> > > Thanks in advance for any help.
> > > -Kumari
> >
> > The answer depends very much upon what you mean by the question. Win32
> > has "events", which can be used to accomplish similar things, so the
> > answer is clearly "yes". Win32 events, however, behave, in detail, very
> > differently, and are used differently, so the answer is clearly "no".
> > Which answer do you prefer? ;-)
> 
> Good answer, Dave.  This is one of the most frustrating things about the
> Win32 threading API.  CV's are incredibly powerful and fairly easy to use,
> but Win32 unfortunately ommited them.
> 
> In WinNT 4.0, there is a new API called SignalObjectAndWait which can be used
> to implement a CV pretty easily.  There are two problems:
> 
> (1) This API is not available on WinNT 3.51 or Win95.  Hopefully it will show
> up in Win97, but I don't know for sure.
> 
> (2) Using this API with a mutex and an auto-reset event, you can create a
> CV-lookalike where PulseEvent will behave like pthread_cond_signal, but there
> is no way to immitate pthread_cond_broadcast.  If you use a mutex and a
> manual event, PulseEvent will behave like pthread_cond_broadcast, but there
> is no way to immitate pthread_cond_signal.  (Sigh ...)
> 
> I know ACE has a Win32 CV that works in general, but I seem to recall Doug
> Schmidt saying that it's very complex and not very efficient.
> 
> Ian
================================================================
 Q87: What if a cond_timedwait() times out AND the condition is TRUE?  

  [This comment is phrased in terms of the JAVA API, but the issues are the
same. -Bil]

> After thinking about this further even your simple example can fail.
> Consider this situation. A number of threads are waiting on a condition,
> some indefinitely, some with timeouts. Another thread changes the
> condition, sets your state variable and does a notify(). One of the waiting
> threads is removed from the wait-set and now vies for the object lock. At
> about the same time a timeout expires on one of the other waiting threads
> and it too is removed from the wait-set and vies for the lock - it gets the
> lock! This timed-out thread now checks the state variable and wrongly
> concludes that it received a notification.
> 
> In more complex situations where there are multiple conditions and usage of
> notifyAll() even more things can go wrong.

  You are correct with everything you say, right up until the very last word.
The behavior is NOT wrong.  It may not be what you *expected*, but it's not
wrong.  This is a point that's a bit difficult to get sometimes, and it drives
the real-time crowd to distraction (as well it should), but for us time-shared
folks, it's cool.

  When you get a time-out, you've got a choice to make.  Depending upon what
you want from your program, you may choose to say "Timed-out! Signal error!"
or you may choose to check the condition and ignore the time out should it
be true.  You're the programmer.

  A important detail here...  Everything works CORRECTLY.  In particular, if
a thread receives a wakeup, it is removed from the wait queue at that point
and CANNOT subsequently receive a timeout.  (Hence it may take another hour
before it obtains the mutex, but that's OK.)  A thread which times out will
also be removed from the sleep queue and a subsequent wakeup
(pthread_cond_signal()) will be delivered to the next sleeping thread (if
any).

================================================================
 Q88: How can I recover from a dying thread?  


[OK.  So that's not *exactly* the question being addressed here, but that's
the most important issue.  -Bil]

David Preisler wrote:
> 
> I wish to create an *efficient* and *reliable* multi process/multi threaded
> algorithm that will allow many simultaneous readers (for efficiency) to access
> a block of shared memory,  but allows one and only one writer.
> 
> How could a read counter be guaranteed to be always correct even if your read
> thread or process dies???

Sorry to disillusion you, but this is impossible. Remember that you're
talking about SHARED MEMORY, and assuming that SOMETHING HAS GONE WRONG
with some party that has access to this shared memory. Therefore, there
is no possibility of any guarantees about the shared memory -- including
the state of the read-write lock.

You can approximate the guarantees you want by having some third party
record the identity of each party accessing the lock, and periodically
validate their continued existence. You could then, (assuming you'd
coded your own read-write lock that allowed this sort of manipulation by
a third party), "unlock" on behalf of the deceased read lock holder.
Just remember that the fact that the party had DECLARED read-only
intent, by locking for read access, doesn't guarantee that, in the
throes of death, it couldn't have somehow unintentially modified the
shared data. A read-lock really is nothing more than a statement of
intent, after all. And how far do you wish to trust that statement from
a thread or process that's (presumably equally unintentially) blowing
its cookies?

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698       http://www.awl.com/cp/butenhof/posix.html |
\-----------------[ Better Living Through Concurrency ]----------------/

================================================================
 Q89: How to implement POSIX Condition variables in Win32?  

Subject: Re: How to implement POSIX Condition variables in Win32 (LONG)
Douglas C. Schmidt wrote:

In article <5hbiub$pqa@nr1.ottawa.istar.net>,
Alan Conway waiters_ = 0;
  cv->generation_count_ = 0;
  cv->release_count_ = 0;

  // Create a manual-reset Event.
  cv->event_ =
    ::CreateEvent (NULL,  /* no security */
                   TRUE,  /* manual reset */
                   FALSE,  /* non-signalled */
                   NULL); /* unnamed */
}

The following pthread_cond_wait function waits for a condition 
and atomically releases the associated generation_count_;

  cv->waiters++;

  for
  {
    ::LeaveCriticalSection (external_mutex);

    // Wait until the event is signaled.
    ::WaitForSingleObject (cv->event_, INFINITE);

    ::EnterCriticalSection (external_mutex);

    // Exit the loop when the event_>
    // is signaled and there are still waiting
    // threads from this generation that haven't
    // been released from this wait yet.
    if (cv->release_count_ 0
        && cv->generation_count_ != c)
      break;
  }

  --cv->waiters;

  // If we're the last waiter to be notified
  // then reset the manual event.

  cv->generation_count_--;

  if (cv->generation_count_ == 0)
    ::ResetEvent (cv->event_);
}

This function loops until the event_ HANDLE is signaled and at least
one thread from this ``generation'' hasn't been released from the wait
yet.  The generation_count_ field is used incremented every time the
event_ is signal via pthread_cond_broadcast or pthread_cond_signal.
It tries to eliminate the fairness problems with Solution 1, so that
we don't respond to notifications that have occurred in a previous
``generation,'' i.e., before the current group of threads started
waiting.

The following function notifies a single thread waiting on a Condition
Variable:

int
pthread_cond_signal (pthread_cond_t *cv)
{
  if (cv->waiters_ cv->release_count_)
    {
      ::SetEvent (cv->event_);
      cv->release_count_++;
      cv->generation_count++;
    }
}

Note that we only signal the Event if there are more waiters than
threads currently being released.

Finally, the following function notifies all threads waiting on a
Condition Variable:

int
pthread_cond_broadcast (pthread_cond_t *cv)
{
  if (cv->waiters_ 0)
    {
      ::SetEvent (cv->event_);
      cv->release_count_ = cv->waiters_;
      cv->generation_count_++;
    }
}

Unfortunately, this implementation has the following drawbacks:
 
1. Busy-waiting -- This solution can result in busy-waiting if the
waiting thread has highest priority.  The problem is that once
pthread_cond_broadcast signals the manual reset event_ it remains
signaled.  Therefore, the highest priority thread may cycle endlessly
through the for loop in pthread_cond_wait.

2. Unfairness -- The for loop in pthread_cond_wait leaves the critical
section before calling WaitForSingleObject.  Thus, it's possible that
another thread can acquire the external_mutex and call
pthread_cond_signal or pthread_cond_broadcast again during this
unprotected region.  Thus, the generation_count_ will increase, which
may fool the waiting thread into breaking out of the loop prematurely
and stealing a release that was intended for another thread.

3. Potential for race conditions -- This code is only correct provided
that pthread_cond_signal and pthread_cond_broadcast only ever called
by a thread that holds the external_mutex.  That is, code that uses
the classic "condition variable signal" idiom shown above will work.

Doug
--
Dr. Douglas C. Schmidt                  (schmidt@cs.wustl.edu)
Department of Computer Science, Washington University
St. Louis, MO 63130. Work #: (314) 935-4215; FAX #: (314) 935-7302
http://www.cs.wustl.edu/~schmidt/
================================================================
-- 
================
Bil@LambdaCS.com

================================================================
 Q90: Linux pthreads and X11  

Date: Mon, 16 Feb 1998 17:47:03 +0000
Organization: Visix Software

Steve Cusack wrote:

> I've just started using Linux pthreads and have immediately run into
> the fact that the package appears to be incompatible with X.  I've
> read (via DejaNews) that X is "threads unsafe" and that others have
> had similar problems (X refusing to start).  Does anyone have X11 and
> pthreads working together on a Linux system?  If so, what did you have
> to do?

I ported Vibe (a multi-threaded Java IDE) to Linux, so I have made X11 and
linuxThreads work together.  Its not easy, unless you can target a glibc2
platform.

You need to do one of two things: either get/create a recompilied
-D_REENTRANT version of the X libraries, or patch linuxThreads to use the
extern int errno as the errno for the initial thread.  I chose to create
thread aware Xlibs.  (That was not fun.  For some reason the build process
would get into an infinite loop.  I don't remember how I got it to work)

You should be able to search Deja-news for pointers to patching
LinuxThreads.

Doug
-- doug@xyzzy.com
Replace "xyzzy" with Victor India Sierra India X-Ray to email me.

		================

Get yourself a copy of Redhat 5.0. It comes with pthreads and thread
safe X libs.

	-Arun

		================

For such-compiled binaries for ix86-libc5, see

  ftp://ftp.dent.med.uni-muenchen.de/pub/wmglo/XFree86-3.3-libs.tar.gz

Strange, for me it was very easy.  Just adding -D_REENTRANT next to
-D_POSIX_SOURCE in linux.cf did the trick.

But remember:  These libs are still not thread-safe (you can make only
one X11 call at a time -- I don't think this is too bad).

The better option at this stage really is glibc2 with X11 libs
compiled with XTHREADS.

Regards,
Wolfram.
================================================================
 Q91: One thread runs too much, then the next thread runs too much!  

[I've seen variations on this concern often.  Johann describes the problem
very well (even if he can't find the "shift" key).  -Bil]

===================  The Problem  ================================
Johann Leichtl wrote:
> hi,
>
> i have a a problem using pthreads in c++.
>
> basically what i want to do is have class that manages a ring buffer and
> say 2 threads, where one adds entries to the buffer and one that removes
> them.
> i have a global object that represents the buffer and the member
> functions make sure that adding and removing entries from the buffer
> work ok.
>
> the 2 functions:
>
> inline void ringbuf::put(int req)
> {
>   pthread_mutex_lock(&mBufLock);
>   while(elemNum == size)
>     pthread_cond_wait(&cNotFull, &mBufLock);
>   buf[next] = req;
>   next = (next + 1) % size;
>   elemNum++;
>   pthread_cond_signal(&cNotEmpty);
>   pthread_mutex_unlock(&mBufLock);
> }
>
> inline void ringbuf::get(int& req)
> {
>   pthread_mutex_lock(&mBufLock);
>   while(elemNum == 0)
>     pthread_cond_wait(&cNotEmpty, &mBufLock);
>   req = buf[last];
>   last = (last + 1) % size;
>   elemNum--;
>   pthread_cond_signal(&cNotFull);
>   pthread_mutex_unlock(&mBufLock);
> }
>
> now my problem is that my consumer thread only wakes up after the buffer
> is full. i tried different buffer sizes and simulated work in both
> producer and consumer.
> when i use a sleep() function in the producer (and or consumer) i can
> get the thing to look at the buffer earlier.
>
> i was wondering if anybody would have some input on what the problem
> could be here. i've done something similar with UI threads and not C++
> and it works fine.
>
> thanks a lot.
>
> hans
> hans@brc.uconn.edu

===================  The Solutions  ================================

Use either threads w/ system scope or call thr_setconcurrency to
increase the concurrency level...

Here's a handy thing to stick in a *Solaris* pthreads program:

#include 
#include 

        thr_setconcurrency(sysconf(_SC_NPROCESSORS_ONLN)+1);

This will give you as much actual concurrency as there are processors
on-line plus one.  It's a starting point rather than a fix-all, but
will cure some of the more obvious problems...

- Bart

--
Bart Smaalders                  Solaris Clustering      SunSoft
barts@cyber.eng.sun.com         (415) 786-5335          MS UMPK17-201
http://playground.sun.com/~barts                        901 San Antonio Road
                                                        Palo Alto, CA
94303

	====================================================
Johann,

  No, actually you *don't* have a problem.

  Your program works correctly, it just happens to work in a slightly
unexpected fashion.  The functions that call put & get are probably
unrealistically simple and you are using local scheduling.  First the
buffer fills up 100%, then it completely empties, then it fills up
again, etc.

  Make your threads system scoped and you'll get what you expect.
[You'll notice Bart suggests a different method for obtaining the
same results (ie. more LWPs).  I like this method because I think
it's a clearer statement of intention AND PROCTOOL will give me
LWP statistics, but not thread statistics.]


  (You can look on the web page below for exactly this program, written
in C, one_queue_solution.c.)

-Bil
================================================================
 Q92: How do priority levels work?  

Kamal Kapila wrote:
> 
> Hi there,
> 
> I'm working on an internal package to provide platform independant
> thread services (the initial platforms are DECUNIX 4.0 and Windows NT).
> The problem I'm having is understanding the thread scheduling on
> DECUNIX.
> 
> It would seem to me logical that the threads of a process would have the
> same priority and policy of their associated process by default.
> However, when I check the process priority/policy I get completely
> different values from when I check the individual thread priorities and
> policies.  In fact, the priority values do not seem to even follow the
> same scale (I can set process priorities from 0-63, while thread
> priorities go only from 0-31).  In addition, setting the process
> priority does not seem to effect the thread priorities at all (!).

Basically, there are "interaction issues" in implementing a 2-level
scheduling model (as in Digital UNIX 4.0), that POSIX didn't attempt to
nail down. We deferred dealing with these issues until some form of
industry concensus emerged. That industry concensus has, since, not
merely "emerged", but has become a mandatory standard in the new Single
UNIX Specification, Version 2 (UNIX98).

With 2-level scheduling, it really doesn't make much sense to "inherit"
scheduling attributes from the process -- because those attributes MEAN
entirely different things. Digital UNIX, by the way, doesn't really have
a "process" -- it has (modified) Mach tasks and threads. (There are a
set of structures layered over tasks to let the traditional UNIX kernel
code deal with "processes" in a more or less familiar way, but a process
is really sheer illusion.) Since tasks aren't scheduled, they really
have no scheduling attributes -- threads do. Since a non-threaded
process has a task and a single thread, the 1003.1b (realtime)
scheduling functions operate, in general, on the "initial thread" of the
specified "process".

The kernel thread scheduling attributes control scheduling between
various kernel threads. But a POSIX thread is really a user object, that
we map onto one or more kernel threads (which we call "virtual
processors"). Pretending to set the scheduling attributes of this thread
to the "process" attributes makes no sense, because the scheduling
domain is different. POSIX threads are scheduled only against other
threads within the process -- not against kernel threads in other
processes.

POSIX provides a way to create threads that you really want to be
scheduled against other kernel threads -- essentially, forcing the POSIX
thread to be "bound" to a kernel thread itself, at the expense of (often
substantially) higher scheduling costs. This is called "system
contention scope". Digital UNIX 4.0 didn't support system contention
scope (which is an optional feature of POSIX), but we've added it for
the next version (4.0D).

Each system contention scope (SCS) thread has its own scheduling
attributes, independent of the process. While it might make some
intuitive sense to inherit the process priority, POSIX doesn't provide
any such semantics. A newly created thread either has explicit
scheduling attributes, or inherits the attributes of the thread that
created it. Of course, since setting the "process" attributes affects
the initial thread, threads that IT creates will inherit the "process"
attributes by default. But changing the "process" attributes won't (and
shouldn't) affect any SCS threads in the process.

The ambiguity (and the only relevant question for the implementation
you're using, which doesn't support SCS threads), is, what happens to
the virtual processors that are used to execute POSIX threads, when the
"process" scheduling attributes are changed? And with what attributes
should they run initially? UNIX98 removes the (intentional) POSIX
ambiguity by saying that setting the 1003.1b scheduling attributes of
the "process" WILL affect all "kernel entities" (our virtual processors,
Sun's LWPs) used to execute process contention scope (PCS, the opposite
of SCS) threads. By extension, the virtual processors should initially
run with the existing process scheduling attributes.

This will be true of any UNIX98 branded system -- but until then,
there's no portable rules.

The fact that the POSIX thread interfaces don't use the same priority
range as the system is a stupid oversight -- I just didn't think about
it when we converted from DCE threads to POSIX threads for 4.0. This has
been fixed for 4.0D, though it's a bit too substantial a change (and
with some potential risk of binary incompatibilities) for a patch.

> (BTW, I am using sched_getparam() and sched_getscheduler() to get the
> process related values and  pthread_getparam() to get the thread related
> values).

Right.

> Specifically, I have the following questions :
> 
> - What is the relationship between the process priority/policy and the
> thread priority and policy  ?

There's very little relationship. Each POSIX thread (SCS or PCS) has its
own scheduling attributes (priority and policy) that are completely
independent of "process" attributes. UNIX98, however, says that the
"kernel entities" used to execute PCS POSIX threads WILL be affected by
changes to the "process" scheduling attributes -- but SCS threads will
not (and should not) be affected by such changes. (Nor will the
scheduling attributes of PCS threads, even though their "system
scheduling attributes" effectively come from the virtual processor,
which is affected.)

> - Does the scheduler schedule individual threads independently, or are
> processes scheduled, with a process's threads then sharing the process
> CPU time?

As I said, there's no such thing as a process, and the closest analog,
the Mach task, isn't a schedulable entity. All threads are scheduled
independently -- each has its own scheduling attributes, its own time
slice quantum, etc. On Digital UNIX 4.0, with only PCS threads, the
kernel schedules the virtual processor threads of all the processes
(plus the single kernel threads associated with all non-threaded
processes). Threaded processes also contain a user-mode scheduler, which
assigns PCS threads to the various virtual processors, based on the PCS
thread scheduling attributes. (A process has one virtual processor for
each available physical processor on the system.)

On Digital UNIX 4.0D, with SCS thread support added, each process may
also have any number of SCS threads, which map directly to individual
and independent kernel threads. SCS threads are scheduled the same as
virtual processors -- each has its own scheduling attributes, time slice
quantum, etc.

(It might seem that managing CPU time by kernel threads rather than by
processes allows users to monopolize the system by creating lots of
kernel threads. But they could do that by creating lots of processes,
too... and a kernel thread is cheaper for the system than a process,
which is really a thread plus a task. The ability to create new kernel
threads, as well as processes, is limited both by user and system
quotas. And of course, in 4.0, users can't actually create new kernel
threads -- only POSIX threads, which are mapped to the process' existing
virtual processors.)

So each process presents a set of runnable kernel threads to the kernel:
A mix of SCS threads and the various PCS threads currently mapped on to
one or more virtual processors. The kernel then determines which kernel
threads to schedule on each processor. (That's why it's called "2-level
scheduling".)

> - Is the thread's overall priority a combination of the process priority
> and the individual thread priority ? If so, how is this determined ?

Currently, "process" priority is irrelevant for a threaded process.
Virtual processors don't inherit the process priority. (Actually, they
sort of do, and the first virtual processor is the initial process
thread, which can be changed using the 1003.1b functions -- but the
kernel generates "replacement" virtual processors at various times, and
these currently are always set to the default scheduling attributes
[timeshare policy and priority 19].)

POSIX thread priority determines which threads the user-mode scheduler
assigns to the various virtual processors. Because the virtual processor
priority doesn't change (the whole point of 2-level scheduling is to
avoid expensive kernel calls), the POSIX thread priority has no effect
on the kernel scheduling. That's OK, except in rare cases where
applications comprising multiple PROCESSES have threads (in different
processes) that really need to directly preempt each other based on
priority.

> I have read through all of the Digital documentation that I have but I
> have not been able to find any clear answers to my questions.

A description of the behavior (though in less technical/internal detail
than the one in this posting) can be found in Appendix A (section A.3)
of the Digital UNIX "Guide to DECthreads" manual.

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698       http://www.awl.com/cp/butenhof/posix.html |
\-----------------[ Better Living Through Concurrency ]----------------/
================================================================
 Q93: C++ member function as the startup routine for pthread_create().  



Hi
You need a static member function
	static void *function_name(void  *);
in the class declaration

Usually then you pass the classes address as the parameter so the
static function can access the nonstatic members of the class

- Robert

On 19 Sep 1997 03:15:22 GMT, romig@cse.unl.edu (Phil Romig) wrote:
>
>
>I know I should be able to figure this out, but I'm missing something.
>
>I want to pass a member function as the startup routine for pthread_create().
>That is, I want to create an instance of a class and then pass one of
>the public member functions as the start routine to pthread_create().
>
>I believe the question comes down to how describe the address of the
>member.  Simple qualification (class::member) will not work because I
>need the address of the of the function that goes with the particular
>instance of the class.
>
>For the record I'm working on an HPUX 10.01 system so I'm using pthreads
>draft 4, rather than the current standard.  
>
>Any advice, pointers and suggestions are welcome.
>thanks
>Phil
>romig@cse.unl.edu
>
>
>A quick example of what I want to try:
>
>  class foo {
>  public:
>   foo(int i)
>   void *go(void *arg);
>  }
>
> main() {
>  foo *bar = new foo(1);
>
>  pthread_create(...,&(bar->go),....);
>}
================================================================
 Q94: Suprious wakeups, absolute time, and pthread_cond_timedwait() 

[Bil: The summary is "Retest conditions with CVs." and "The time-out is an
absolute time because it is."  (NB: Deltas are a proposed extension to POSIX.)
This is a nice exposition.]

Brian Silver wrote:
> 
> Ben Self wrote:
> >
> [Snip]
> > The standard specifies that pthread_cond_wait() and
> > pthread_cond_timedwait() may have spurious wakeups.  The reason for this
> > is that a completly reliable once and only once wake up protocol can be
> > excessively expensive for some asymetric multiprocessor systems.
> 
> Well, maybe I'm being a bit anal about this, but this
> really isn't the case. If it was, then you'd have the
> same issue for mutexes as well, and the standard does
> not allow for spurious wakes on mutexes.

[Bil: Actually, you DO get spurious (define "spurious"!) wakeups with mutexes,
YOU just never see them.]
 
> The "while(predicate)/wait" construct is very common
> in concurrent environments (regardless of their symetry).
> The reason is that since the environment is highly
> unpredictable, while you were coming back out of the
> wait, the state of the thing that you were waiting for
> may have changed.
> 
> This construct is used to impliment mutexes as well,
> its just that you don't see it since the predicate
> is known; it is the state of the mutex lock. Cv's force
> the construct to the user code because the predicate
> is not known to the impliment of the cv itself. The
> warnings about spurious wakes are taken seriously when
> mutexes are implimented, and are accounted for in the
> exact same "while(predicate)/wait" construct.
> 
> Wake-only-once doesn't really help. It will remove the
> addition of spurious wakes, but it won't account for
> "valid wake, but the predicate changed". Implimenting
> wake-only-once is expensive when you consider that this
> solution solves both problems.
> 
> Also note that the mutex lock around the predicate
> doesn't solve this problem either. There is a race
> that starts once you see the wake and ends once you
> reaquire the mutex. In that time, another thread can
> get the mutex and change the data (believe me, it
> happens - more often than you'd expect). When you
> reaquire the mutex, and exit the wait, the predicate
> has changed and you'll need to go back to waiting.
> 
> Now, a wake-only-once,-and-gimme-that-mutex atomic
> operation might be nice .
> 
> Brian.

I am not  reposting to be defensive or argumentative.  Upon reflection,
however, I have come to the conclusion that neither I nor subsequent
posters have really dealt with the original poster's question let alone
the new topics that we have thrown about. 

Since this is a response largely to Brian Silver's post, a person I have
a good deal of respect for, I have chosen to include some quotes form
Dave Butenhof's book, Programming with POSIX Threads, because we both
know it and have a mutual admiration for his work.

First of and most importantly the original question I attempted to
answer was:

Fred A. Kulack wrote:
> A side question for the rest of the group...
> All the applications I've seen use a delta time for the wait and must
> calculate the delta each time a cond_timedwait is done. What's the rational
> for the
> Posix functions using an ABSOLUTE time?

I was hoping for an uncomplicated answer and the spurious wakeup issue
seemed to fit the bill.  My writing and thinking however was too
simplistic to provide any meaningful insight.  So I will try again. 
Please realize that some of what you will read below is purely personal
supposition.  Chime in if I have misinformed.

1)  Spurious wakeup is a reason for passing a absolute value to
cond_timedwait.  It is not the reason or even a particularly important
reason.  The standard (POSIX 1003.1c-1995) specifically states that a
compliant implementation of pthread_cond_timedwait() may suffer from
spurious wakeups.  It therefore is reasonable to use and absolute
timeout value instead of an delta to simplify the act of retry.

2)  More importantly it is also very likely a performance issue.  Most
systems when scheduling a software interrupt use an absolute value that
reflects an offset into the OS's epoch.  To constantly be re-evaluating
a delta in user code is excessively expensive especially if most systems
really want an absolute value anyway.

3)  Also their is the reality that the structure timespec is the high
resolution time value of choice in POSIX.  And timespec happens to
represent its time as absolute time.  Add into that the needs of the
powerful realtime group that had a great impact of the shape of POSIX
1003.1c.   What integral unit would we use for a delta anyway? and would
it be in nanoseconds?  Eeak!

4)  Most importantly one would hope that the interface were constructed
to promote good coding techniques.  As Brian Silver stated the
"while(predicate)/wait" idiom is an important technique for far more
reasons than just spurious wakeups.  By using an absolute timeout value
as opposed to a delta this idiom is directly supported by easing its
use.

When I originally brought up the  "while(predicate)/wait" idiom it was
because spurious wakeups would necessitate retrying the predicate.  I
did not intend to state that this was the only or even a particularly
important reason for the pattern.  The while "while(predicate)/wait"
idiom or an equivalent is essential to programming with condition
variables.

1)  Most importantly is the reason Brian silver stated, "There is a race
that starts once you see the wake and ends once you reacquire the
mutex."  It would be difficult and detrimental to concurrency to
construct through synchronization a situation that did not require
re-testing of the predicate after a wakeup.  This is why Brian's magic
bullet "wake-only-once,-and-gimme-that-mutex atomic" does not exist. 
Although it would be nice.

2)  Spurious wakeups do exist. Be consoled by the fact that "The race
condition that cause spurious wakeups should be considered rare.
[Butenhof]"

3)  Also It enables a powerful technique that I have been using for a
several years with great success that Dave Butenhof refers to as "loose
predicates".  "For a lot of reasons it is often easy and convenient to
use approximations of actual state.  For example, 'there may be work'
instead of 'there is work'."  I will go one step beyond that in my
experience of coding distributed web servers there are situations when
the notification mechanism cannot know with certainly that there is work
without actually have performed the entirety of the task itself.  Often
the best a distributed component has to work with is trends and
potentialities.

Lastly, (whew ;) I believe that I have overstated the significance of
the performance implications of only once wakeups.  "Excessively
expensive" is a bit strong without further qualification.  If it were
such a paramount issue Brian Silver is right, mutexes would suffer from
the same restrictions and they absolutely do not.  

There is a performance issue that I have run across many times and have
seen cited in many references including : "Spurious wakeups may sound
strange but on some multiprocessor systems, making condition wakeup
completely predictable might substantially slow all condition variable
operations. [Butenhof]"  Never-the-less, it is the fact that making
wakeup completely predictable does not get you that much.  You still
need to retest your predicate.  In the end it is such an easy and cheap
thing when taken in the context of the overhead of the synchronization
and latency of the wait.

--ben


-----------
Ben R. Self
bself@opentext.com

www.opentext.com
Open Text Corporation -- Home of Livelink Intranet
================================================================
 Q95: Conformance with POSIX 1003.1c vs. POSIX 1003.4a? 

christof ameye we41 xxxx wrote:

> Some pthread libraries talk about conformance with POSIX 1003.1c and
> others of conformance with POSIX 1003.4a. What are the
> differences/similarities ?
> I don't realy need the answer, but it might be interesting to know ...

First off, "conformance" to 1003.4a is a completely meaningless
statement. There's no such thing, because 1003.4a was never a standard.
Furthermore, I strongly doubt that there ever *were* any implementations
that could conform even if there was a way to apply meaning to the
statement.

1003.4a was the original name of the thread standard -- named for the
fact that it was developed by the realtime group (POSIX working group
designation 1003.4). However, it, like the original realtime standard,
was really an amendment to the "base standard", 1003.1. Eventually,
POSIX decided to resolve the confusion by creating more -- and renaming
all O/S API standards into the 1003.1 space. Thus, 1003.4 became
1003.1b, 1003.4a became 1003.1c, 1003.4b became 1003.1d, and so forth.

There were various draft versions of the standard while it was still
named 1003.4a, but all are substantially different from the actual
standard, and to none of them, technically, can any implementation
"conform". The most common draft is 4, which was the (loose) basis for
the "DCE thread" api, part of The Open Group's DCE suite. There's at
least one freeware implementation that claimed to be roughly draft 6.
IBM's AIX operating system provides a draft 7 implementation. The
"pthread" interface on Solaris 2.4 was draft 8. There is also at least
one implementation which I've seen claiming to be "draft 10". Draft 10
was the final draft, which was was accepted by the IEEE standards board
and by ISO/IEC with only "minor" editorial changes. Nevertheless, draft
10 is NOT the standard, and, technically, one cannot "conform" to it.
"Draft 10" and "1003.1c-1995" are NOT interchangeable terms.

Finally, because 1003.1c-1995 was never published as a separate
document, the official reference is the 1003.1-1996 standard, which
includes 1003.1b-1993 (realtime), 1003.1c-1995 (threads), and
1003.1i-1995 (corrections to the realtime amendment).

In terms of someone writing programs, a lot of that is irrelevant. But
you need to be aware that there's no real definition of "conformance"
for any drafts, so one vendor's "draft 4" is not necessarily the same as
another's "draft 4", and, while that might be inconvenient for you,
there's nothing "wrong" with it. (Although, from the POSIX standards
point of view, it was foolish and irresponsible of "them" [by which I
really mean "us", since I wrote the most common draft 4 implementation,
the original DCE threads reference library ;-) ] to use the "pthread"
prefix at all.)

There are MANY differences between "draft 4" and standard POSIX threads.
There are many (though slightly fewer) differences between draft 7 or 8
and the standard. There are even some differences between draft 10 and
the standard. Look at a move between any two drafts, or between any
draft and the standard, as a PORT to an entirely new threading library
that has some similarities. Be very careful of the details, especially
where things "appear to be the same". And, if you're stuck with a draft
implementation, lobby the vendor to provide a full conforming POSIX
implementation!

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698       http://www.awl.com/cp/butenhof/posix.html |
\-----------------[ Better Living Through Concurrency ]----------------/

================================================================
 Q96: Cleaning up when kill signal is sent to the thread.? 


> I'm writing a multi-threaded daemon, which requires some cleanup if a
> kill signal is sent to the thread.  I want just the thread that received
> the signal to exit.
> 
> The platform is Linux 2.0, libc 5.4.23, linuxthreads 0.6 (99% POSIX
> threads).
> 
> The docs indicate that threads share signal functions, but can
> individually block or accept certain signals.  This is workable -- but
> how do I get the thread id of the thread that received the signal?
> 
> And my next question, how portable are thread cleanup routines?
> 
> Thanks,
> 
> Jeff Garzik                                        Quality news feeds
> News Administrator                     INN Technical info, Consulting
> Spinne, Inc.                            http://www.spinne.com/usenet/

Jeff,

  From the sounds of what you say, the answer is "No."  :-)

  Meaning, don't do that.  There's a better method, cancellation.
If you really want the thread to exit asynchronously, that's the
way to do it.

  Now, it is even more likely that a simple polling routine will
do the job, and that would be even easier to write.

-Bil
(There's a nice little cancellation example on the web page below.)

================================================================
 Q97: C++ new/delete replacement that is thread safe and fast? 

> bobp@troi.erols.com (Bob Pearson) writes:
> 
> > Our platform is Solaris 2.5.1 and I am looking for a commerical, freeware
> > or shareware C++ new/delete replacement that is thread safe and uses more
> > than a single mutex.  We have a multi-threaded application that is using a
> > tremendous amount of new operators and is huge (>200MB) and is constantly
> > running into very high mutex contention due to the single mutex for new in
> > libC.a from:
> >
> >       SUNWSpro "CC: SC4.0 18 Oct 1995 C++ 4.1".
> 
> You might want to check out ptmalloc:
> 
>    ftp://ftp.dent.med.uni-muenchen.de/pub/wmglo/ptmalloc.tar.gz
> 
> I would hope that operator new somehow invokes malloc at a lower
> level.  If not, you would have to write a small wrapper -- there
> should be one coming with gcc that you could use.
> 
> Hope this helps,
> Wolfram.
Wolfram Gloger 

================================================================
 Q98: beginthread() vs. endthread() vs. CreateThread? (Win32) 

[Bil: Look at the description in "Multithreading Applications in Win32" (see
books.html)]

Mark A. Crampton wrote:
> 
> Juanra wrote:
> >
> > I'm a Windows 95 programmer and I'm developing a multithreaded
> > server-side application. I use the CreateThread API to create a new
> > thread whenever a connection request comes. I've read that it's better
> > to use beginthread() and endthread() instead of CreateThread because
> > they initialize the run time libraries. What happens with win32
> > CreateThread function?. Doesn't it work properly?. If not, I can't use
> > beinthread because I can't create my thread in a suspended mode and
> > release it after.
> >
> > Does the function beginthreadNT() work under win95?
> 
> No
> 
> >
> > Thanks in advance.
> > Juan Ra.
> 
> Answer to beginthread - use _beginthreadex, which uses same args as
> CreateThread (you can create suspended).  _beginthreadex works on 95 &
> NT but not Win32S.  The priviledge flags are ignored under 95.
> 
> CreateThread _works_ OK - it just doesn't free memory allocated on the C
> run-time library stack when the thread exists.  So you can attempt to
> clean up the runtime library stack, use _beginthreadex, or not use any C
> run time library calls.

================================================================
 Q99: Using pthread_yield()? 

Johann Leichtl wrote:
> 
> if i have some code like:
> 
>         ..
>         pthread_yield()
>         something(e.g. lock mutex)
>         ..
> 
> is it guaranteed that the thread will give up the cpu before getting the
> lock or not.

First off, to clarify, (you probably already know this, given the set of
names in your subject line), "pthread_yield" is an obsolete DCE thread
interface, not part of POSIX threads. As such, it is not covered by any
formal standard, and has no real portability guarantees. The way it
works on your particular DCE thread system is probably the way the
developers wanted it to work on that system, and if you disagree there's
no "higher authority" to which you might appeal.

POSIX specifies the behavior of sched_yield, (or, in fact, any
scheduling operation), only with respect to the defined realtime
scheduling policies, SCHED_FIFO and SCHED_RR. Threads running under one
of these policies that call sched_yield will release the CPU to any
thread (in SCHED_FIFO or SCHED_RR) running at the same priority. (There
cannot be any at a higher priority, since they would have preempted the
current thread immediately.)

Is that the same thing as "guaranteed [to] give up the cpu"? For one
thing, sched_yield won't do anything at all if there are no other
threads that are ready to run at the calling thread's priority; it'll
just return.

If you have threads with non-standard scheduling policies, such as
SCHED_OTHER, or a hypothetical SCHED_TIMESHARE, POSIX says nothing about
the behavior or sched_yield. Most likely, (and at least in Digital's
implementation), the function will do the same thing. It doesn't really
worry about scheduling POLICY, only PRIORITY. Note that, because
SCHED_OTHER doesn't necessarily imply preemptive scheduling, you might
actually have a thread "ready to run" at a higher priority than the
current thread's priority. Also, because non-realtime policies aren't
necessarily "strictly priority ordered", and the system generally wants
to simulate some sort of fairness in timeshare scheduling, it is
possible (at least, "not ruled out by the standard") that a call to
sched_yield from a non-realtime thread might yield to a thread with
lower priority -- especially if that other thread is realtime.

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698       http://www.awl.com/cp/butenhof/posix.html |
\-----------------[ Better Living Through Concurrency ]----------------/

================================================================
 Q100: Why does pthread_cond_wait() reacquire the mutex prior to being cancelled? 

Firstly, thanks to all of you who responded to my post in
comp.programming.threads.  As I alluded to in my posting, I felt quite sure
the problem I was experiencing was one due to misunderstanding.  I
immediately suspected this when my program exhibited the same behaviour
under HP-UX *and* Solaris.

Everyone told me the same thing: use a cleanup handler, pushed onto the
cleanup handler stack for the active thread because pthread_cond_wait
*reacquires* the mutex when it is cancelled.  I can see how this causes
other threads waiting on the same condition variable to fail to be
cancelled, but for me, the $64,000 question is:

        Why does pthread_cond_wait reacquire the mutex prior to being cancelled?

This seems like madness to me.  We're _cancelling_ the thread, so we're no
longer interested in the value of the data we're testing.  Why acquire the
lock and immediately require us to use a cleanup handler?

There must be something more to this ;-)

Thanks,
Ben
---
Ben Elliston                    .====    E-mail: ben.elliston@compucat.com.au
Compucat Research Pty Limited  /  ====.     Web: http://www.compucat.com.au
Canberra ACT Australia         .====  /
                                 ====.
Ben,

  You're not thinking hard enough!  It *has* to be like this.

>         Why does pthread_cond_wait reacquire the mutex prior to being cancelled?
> 
> This seems like madness to me.  We're _cancelling_ the thread, so we're no
> longer interested in the value of the data we're testing.  Why acquire the
> lock and immediately require us to use a cleanup handler?

A:	pthread_cond_wait(m, c);
B:	do some work...
C:	pthread_mutex_unlock(m);

  If cancellation happened while sleeping (at A) or while running (at B),
the same cleanup handler would run.  If the state of the mutex was DIFFERENT
at those locations, you'd be up the creek.  Right?

-Bil
================================================================
 Q101: HP-UX 10.30 and threads? 
o
Bryan Althaus wrote:
> 
> Jim Thomas (thomas@cfht.hawaii.edu) wrote:
> : In article <5qj71l$ri1@panix.com> bryan@panix.com (Bryan Althaus) writes:
> :
> : Bryan> This is actually becoming a really bad joke.  No HP people seem to want
> : Bryan> to talk about 10.30 though they will post on what compiler flags to
> : Bryan> use to compile an app using pthreads under 10.30!
> :
> : Bryan> And apparently once it does come out you must ask for it.  10.20 will be
> : Bryan> the shipping OS until HP-UX 11.0 comes out.
> :
> : Bryan> If anyone knows when 10.30 is shipping, please email me.  I'll respect
> : Bryan> your privacy and not repost.  We have a current need for threading,
> : Bryan> but since they must be spread out over multiple CPU's, kernel thread
> : Bryan> support is needed - hence HP-UX 10.30.
> :
> : I received an e-mail from "SOFTWARE UPDATE MANAGER" Saturday that says the
> : following.  Note especially the part about not for workstations :-(
> :
> : Jim
> :
> Thanks for the info Jim.  A received email from a kind soul at HP who
> basically explained the deal on 10.30, being used for people/ISV
> transitioning for HP-UX 11.0, but was not sure when it would be out.
> That was all I needed to know. Based on this we will use 10.20 for
> our product roll-out and when HP-UX 11.0 comes out maybe I revisit
> replacing the forking() code with threads.  As it turned out, on
> a two CPU machine, the forking() code actually worked nicely and
> basically was written as if I were using pthreads and wasn't really
> the big hack I thought it was going to be.  Of course each fork()
> costs an additional 70MB's of memory! :)
> 
> Now if someone could let us know roughly the due date for HP-UX 11.0
> and maybe what we can look for in HP-UX 11.0.  Obviously it will have
> kernel threads with pthreads API, NFS PV3, Streams TCP/IP, and support
> both 32 and 64 bit environments. Will HP-UX 11.0 ship more Net friendly
> with a Java Virtual Machine?  Will the JVM be threaded? Will JRE be
> on all HP-UX 11.0 systems?  WebNFS support? WebServer? Browser?  Current
> OS's now come with all these goodies standard: http:/www.sun.com/solaris/new

-- 
================================================================
 Q102: Signals and threads are not suited to work together? 

Keith Smith wrote:
> 
> This is a question I posed to comp.realtime, but noticed that you have a
> discussion going on here....  can you offer me any assistance?
> HEre's the excert:
> 
> Shashi:
> 
> Based on your previous email (below), I have a couple of questions:
> 
> 1. If signals and threads are not suited to work together, what
> mechanism can/should be used to implement timing within a thread.  If I
> have two threads that performed autonomous time-based functions, I want
> to be able to have a per-thread timing mechanism.
> 
> 2. If the approach of "block all signals on all threads and send various
> signals to a process - putting the emphasis on the thread to unblock the
> appropriate signals", how do we deal with other threads which may be
> interrupted by a blocked signal (e.g. a read() call that returns EINTR
> even when its thread blocks the offending signal.  Isn't this a flaw?
> This requires the need for a signal handler (wastefull) with the RESTART
> option speicified.
> 
> It seems like a per-thread mechanisms is needed... how does NT
> accomplish this?
> 
> ** I know I shouldn't be relying on a per-LWP signal, but how else can I
> accomplish what I am trying to do?
> 
> In message " timer's interrupting system calls... HELP", Shashi@aol.com
> writes:
> 
> >Hi,
> >Signals and threads are not suited to work together. Personally, I feel that
> >UNIX has a serious flaw in the sense that most blocking systems calls (e.g.
> >read, semop, msgrcv etc) do not take a timeout parameter. This forces
> >programmers to use alarms and signals to interrupt system calls. I have
> >worked with operating systems such as Mach, NT and other which do not suffer
> >from this problem. This makes porting single threaded applications from UNIX
> >(which rely on signals) to a multithreaded process architecture difficult.
> >
> >Even though I come from an UNIX background (Bell Labs in late 80's) I have
> >learnt the hard way that signals make program much more error prone. I have
> >worked extensively on Mach and NT and never saw a reason to use threads. As
> >far as POSIX.1c is concerned I think they did a favor to the users of threads
> >on UNIX by mandating that signals be a per-process resource. You have to
> >understand that LWP is more of a System R4 concept (same on Solaris) and not
> >a POSIX concept. Two level scheduling is not common on UNIX systems (those
> >who implement have yet to show a clear advantage of two level scheduling).
> >
> >I am sure that Dave Butenhof (frequent visitor to this newsgroup) would have
> >more insight as to why POSIX did not choose to implement signals on a
> >per-thread basis (or LWP as you say). I would advice that you should
> >rearchitect your application not to depend on per-thread (LWP) signals. I
> >feel you will be better off in the long run. Take care.
> >
> >Sincerely,
> >Shashi
> >

================================================================
 Q102: Patches in IRIX 6.2 for pthreads support? 

Jeff A. Harrell wrote:
> 
> radha subramanian wrote:
> >
> > I heard that a set of patches have to be applied in IRIX 6.2
> > for pthreads support.  Could someone tell me which are these
> > patches ?
> 
>  1404 Irix 6.2 Posix 1003.1b man pages          List    123Kb   07/01/97
>  1645 IRIX 6.2 & 6.3 POSIX header file updates  List    41Kb    07/01/97
>  2000 Irix 6.2 Posix 1003.1b support modules    List    164Kb   07/01/97
>  2161 Pthread library fixes                     List    481Kb   07/01/97
> 
> The whole set is downloadable from:
> 
> 
> 
> A SurfZone password is required.


================================================================
 Q104: Windows NT Fibers? 

Ramesh Shankar wrote:
> 
> Found some info. on Windows NT Fibers in "Advanced Windows." Just
> wanted to verify whether my understanding is correct.
> 
> - Is a (primitive) "many to one" thread scheduling model.
> - Fibre corresponds to Solaris "threads" (NT threads then correspond
> to Solaris LWP).
> - If a fibre blocks, the whole thread (LWP for us) blocks.
> - Not as sophisticated as Solaris threads.

  Kinda-sorta.  Certainly close enough.  My understanding is that fibers
were built especially for a couple of big clients and then snuck their way 
out.  As such, I would avoid using them like the plague.  I've read the
APIs and they scare me.

-Bil


================================================================
 Q105: LWP migrating from one CPU to another in Solaris 2.5.1? 

Hej Magnus!

  Kanska...
 
> Hi!
> 
> I've got a question about threads in Solaris 2.5.1, that I hope You can
> answer for me!
> 
> Short version:
> How does the algorithm work that causes an LWP to migrate from one CPU to
> another CPU in Solaris 2.5.1?

  The LWP gets contexted switched off CPU 0.  When a different CPU becomes 
available, the scheduler looks to see how many ticks have passed.  Solaris 2.5:
if less than 4, some other LWP (or none at all!) gets the CPU.  If > 3, then
just put the LWP on the new CPU.
 
> Longer version:
> I'm doing some research about a tool that I hope could be used by multi-thread
> programmers in order to find and possibly correct perfomance bottlenecks.
> Basically the tool works in three phases:
> 1) By running the multi-threaded program on a single processor we create a
>    trace wich represent the behaviour of the program.
> 2) By simulating (or re-schedule) the trace on a multi-processor we can tell
>    wether the program has the desired speed-up or not.
> 3) The simulated "execution" is displayd graphically in order to show where
>    the performance bottlenecks are.

  This sounds good...
 
> I've got a problem when simulating a program that hits a barrier.
> Assume that we, for instance, have 8 bound threads hitting the same barrier
> on a multiprocessor with 7 processors. Here the migration for an LWP from
> one CPU to another is very important. If we have no migration at all the speed
> up will be 4 compared to a single processor.
> On the other hand, if we have full migration, the speed up will be (almost) 7
> if we neglect the impact of cache-misses.

  Of course said $ misses are a BIG deal.

  None-the-less...  I *think* this will happen on Solaris 2.5:

  The first 7 wake up and run for 1 tick (10ms).  The 7 drop 10 points of
priority.  T8 then gets CPU 7, while T1 - T6 run another tick.  They drop 10
points.  T7 wants CPU 7 and will get it from T8.  Now the time slice increases
because we're near the bottom of the priority scale.  Everybody runs for 10
ticks.  From here on out, one thread will migrate around while the others 
keep their CPUs.  I think.

  Of course you'd avoid writing a program that put 8 CPU-bound threads on 7
CPUs...

-Bil
================================================================
 Q106: What conditions would cause that thread to disappear? 

William,

> I have a service thread which enters a never-exiting service loop via
> a while(1).  What conditions would cause that thread to disappear?

  You tell it to.  Either return(), pthread_exit(), or pthread_cancel().
That's the only way out.

> It can't be just returning off the end because of the while(1).  Past
> experience has indicated to me that if a single thread causes a
> exception such as a SEGV that the entire process is killed.  Are there
> known conditions which cause just the thread to exit without
> interfering with the rest of the process?

  You're right.  SEGV etc. kill the process (unless you replace the
signal handler).


> I suspect there's stack corruption in this thread, but I would have
> expected such corruption to take the form of a SEGV or something
> similar.  I'm very surprised that just the thread exited leaving
> everything else (seemingly) intact.

  So...  you have a problem.  I *expect* that you'll find the place
where the thread's exiting and it'll be something you wrote.  (The
other option is a library bug.  Always possible (if unlikely).)

  I'm disappointed to see that a breakpoint in pthread_exit() doesn't
get called in the Sun debugger.  Moreover, you don't even get to 
see the stack from the cleanup handlers!  (I'm making this a bug
report.)  I notice that from TSD destructors you at least get to
see a bit of the call stack.

  So...  I'd suggest this:  Declare some TSD, put a breakpoint in
the destructor, and see what happens when your thread exits.  Try
out the bit of code below.

  How does this work on other platforms?

cc -o tmp1 tmp1.c  -g -lpthread
*/

#define _POSIX_C_SOURCE 199506L
#include 
#define NULL 0

pthread_attr_t 	attr;
pthread_t  	thread;
pthread_key_t  	key;

void destroyer(void *arg)
{pthread_t tid = pthread_self();
 printf("T@%d in TSD destructor.\n", tid);
}


void cleanup(void *arg)
{pthread_t tid = pthread_self();
 printf("T@%d in cleanup handler.\n", tid);
}

void search_sub2()
{ 
 pthread_exit(NULL);		/* Surprise exit -- the one you forgot about */
}

void search_sub1()
{ 
 search_sub2();		/* do work */
}


void *search(void *arg)
{
  pthread_setspecific(key, (void *) 1234);   /* NEED A VALUE! */
 pthread_cleanup_push(cleanup, NULL);
 search_sub1();		/* do work */
 pthread_cleanup_pop(1);
 pthread_exit(NULL);
}


main()
{

  pthread_key_create(&key, destroyer);
 pthread_attr_init(&attr);
 pthread_attr_setscope(&attr, PTHREAD_SCOPE_SYSTEM);
 pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);/* Also
default */

 pthread_create(&thread, &attr, search, NULL);

 pthread_exit(NULL);
}



================================================================
 Q107: What parts, if any, of the STL are thread-safe? 


Matt Austern wrote:
> 
> Boris Goldberg  writes:
> 
> > > >I'm finding a memory leak in the string deallocate() (on the call to
> > > >impl_->deallocate()) under heavy thread load, and it brings up a
> > > >frightening question:
> > >
> > > >What parts, if any, of the STL are thread-safe?
> > >
> > 
> > STL thread safety is implementation-dependent. Check with
> > your vendor. Many implementations are not thread-safe.
> 
> One other important point: "thread safety" means different things to
> different people.  Programming with threads always involves some
> cooperation between the language/library and the programmer; the
> crucial queston is exactly what the programmer has to do in order to
> get well-defined behavior.
> 
> See http://www.sgi.com/Technology/STL/thread_safety.html for an
> example of an STL thread-safety policy.  It's not the only conceivable
> threading policy, but, as the document says, it is "what we believe to
> be the most useful form of thread-safety."

================================================================
 Q108: Do pthreads libraries support cooperative threads? 


Paul Bandler wrote:
> 
> Bryan O'Sullivan wrote:
> >
> > p> Thanks for those who have sent some interesting replies (although
> > p> no-one seems to think its a good idea to not go all the way with
> > p> pre-emptive pthreads).
> >
> > This is because you can't go halfway.  Either you use pthreads in a
> > fully safe manner, or your code breaks horribly at some point on some
> > platform.
> 
> OK, so you would disagree with the postings below from Frank Mueller and
> David Butonhof in July that indicates it is possibe (if inadvisable)?
> 
> Frank Mueller wrote:
> >
> >Raphael.Schumacher@SWISSTELECOM.COM (Schumacher Raphael, GD-FE64) >writes:
> >[deleted...]
> > > 1) Do pthreads libraries support cooperative threads?
> >
> > In a way, somewhat. Use FIFO_SCHED and create all threads at the same priority level,
> > and a thread will only give up control on a blocking operation, e.g. yield, cond_wait,
> > mutex_lock and (if thread-blocking is supported) maybe on blocking I/O (read, write, accept...)
> >
> > This may be close enough to what you want. Short of this, you probably need your coop_yield, yes.
> 
> On HP-UX, at least until 10.30 (which introduces kernel thread support),
> the SCHED_FIFO [note, not "FIFO_SCHED"] scheduling policy workaround
> might work for you, because your threads won't face multiprocessor
> scheduling. I wouldn't recommend it, though -- and of course it won't
> come even close to working on any multiprocessor system that supports
> SMP threads (Solaris, Digital UNIX, IRIX, or even the AIX draft 7
> threads). If you're interested in thread-safety, go for thread-safety.
> While it might be nice to give yourself the early illusion that your
> known unsafe code is running, that illusion could be dangerous later! If
> you've got a real need to run the software sooner than you can convert
> it, you're likely to run into other problems (such as the order in which
> threads run?) If you don't have an immediate need, why look for
> shortcuts that you know are only temporary?
> 
> If you really want to build a "cooperative scheduling" package for your
> threads, (and again, I don't recommend it!), build your own. It's not
> that hard. Inactive threads just block themselves on a condition
> variable until requested to run by some other thread (which signals the
> condition variable and then blocks itself on the same, or another,
> condition variable).
> 
> The "1)" in the original mail implies the first item of a list, but my
> news server has chosen, in its infinitesimal wisdom, not to reveal the
> original post to me. So perhaps I'll have more to say should it repent!
>  
> /---------------------------[ Dave Butenhof ]--------------------------\
> | Digital Equipment Corporation                   butenhof@zko.dec.com |
> | 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
> | Nashua NH 03062-2698       http://www.awl.com/cp/butenhof/posix.html |
> \-----------------[ Better Living Through Concurrency ]----------------/


================================================================
 Q109: Can I avoid mutexes by using globals? 

> > j> Now, I have implemented this without using a synchronization
> > j> mechanism for the integer. Since I have only one writer, and
> > j> multiple readers, and the datum is a simple integer, I believe I
> > j> can get away with this.

> But on the other hand why not do it correctly with locks? Locks
> will make the code easier to maintain because it will be somewhat
> self documenting (the call to rwlock() should give most programmers
> a clue) and it will be more robust. In my experiance threaded
> programs are more fragile and more difficult to debug than single
> threaded programs. It is a good idea to keep thread syncronization
> as controlled as you can, this will make debugging simpler.

Remember that sign in "The Wizard of Oz"?

  "I'd go back if I were you."




When you port your program to the next OS or platform and a new
bug appears...  Could it be caused by your hack?  Won't it be
fun trying to guess with each new bug?  How will you prove to yourself
that the bug is elsewhere?

And that guy who maintains your code, won't he have fun?




That's three opinions...

"Just the place for a snark! I have said it three times.
And what I say thrice is true!"

-Bil

================================================================
 Q110: Aborting an MT Sybase SQL? 


Bryan Althaus wrote:
> 
> Jim Phillips  wrote:
> : We are using Sybase on Solaris for a database application.
> 
> : We are trying to abort a query that turns out to be long running by
> : using POSIX pthread() in a manner which allows a calling X-Windows
> : program to interrupt a fetch loop and cancel the query.
> 
> : At runtime, it works OK sometimes and sometimes it doesn't.  If we
> : attempt to start a new query too soon after canceling the long running
> : query, we get the remains of the old query's result set.  If we wait a
> : couple of minutes before starting the new query,  then it works fine and
> : the new queries expected result set is returned.
> 
> : We are using ESQL and Solaris 2.5 C compiler to build the Sybase SQL
> : Server 11.0.2 program interface.
> 
> : I have heard a rumor that you cannot use pthread with some X-11
> : versions?
> 
> : Anybody out there have any ideas, thoughts, comments or critique(s).
> 
> Having come from a Sybase OpenServer class, I can tell you that you can't
> "cancel" a query.  Once it is sent to Sybase, the query will run till
> completion.  There is no way to stop Sybase from executing the entire
> query.
> 
> I'm not saying that's your problem, just that I notice you say if you
> wait a couple of minutes before starting the new query all is fine.
> Obviously by then the old query has finished.
> 
> I haven't used ESQL in years (use C++/DBtools.h++) so I don't know how your
> connection to Sybase is done. When you "cancel" do you close the old connection,
> and then open up a new connection when you start a new query?
> 
> Just sounds like you may be using the same connection and getting back the
> first query to finish, which would be the "cancelled" one.
> 
> You might try comp.databases.sybase if this theory is at all likely.
> 
> In any case I'd be interested in what the problem turns out to be.
> 
> Good luck,
> Bryan Althaus
> 


================================================================
 Q111: Other MT tools? 


The Etch group at UW and Harvard is distributing some program
development and analysis tools based on the Etch Binary
Rewriting Engine for NT/x86 boxes.

The first of these tools is a call graph profiler that helps 
programmers understand where time is going in their program.
The Etch Call Graph Profiler works on Win32 apps built by most
compilers, does not require source, works with multithreaded
programs, understands program and system DLLs, and is designed
to run reasonably fast even if your machine only has an 'average'
configuration. We've used it on serious programs ranging from
Microsoft's SQL server to Lotus Wordpro, and even on Direct Draw
games like Monster Truck Madness.

If you'd like to give our profiler a try, you can download it from:

	http://etch.cs.washington.edu

Follow the link to Download Etch Tools.

Part of our motivation for distributing these tools is to get
some feedback about what we've built and what we should be
building. So please give our tools a try, and send mail to
etch-support@etch.cs.washington.edu with your comments and
suggestions.

Enjoy!

The Etch Group

	Ted Romer
	Wayne Wong
	Alec Wolman
	Geoff Voelker
	Dennis Lee
	Ville Aikas
	Brad Chen
	Hank Levy
	Brian Bershad
					

================================================================
 Q112: That's not a book. That's a pamphlet! 

Brian Silver wrote:
> Ben Self wrote:
> > 
> [Snip]
> > Since this is a response largely to Brian Silver's post, a person I have
> > a good deal of respect for, I have chosen to include some quotes form
> > Dave Butenhof's book, Programming with POSIX Threads, because we both
> > know it and have a mutual admiration for his work.
> 
> Flattery will get you everywhere!
> 
> (But what makes you think I admire Dave's work!? His code sure is
> formatted nice, though!  He also puts useful comments in
> his code. Except for the thd_suspend implimentation in his book.)

Brian, after all, likes to tell me that I've published a "pamphlet",
because Addison-Wesley chose to use a soft cover, and "books" have hard
covers. For those who may not have figured all this out, by the way,
Brian's office is diagonally across from mine -- he's currently a
contractor working on Digital's threads library.

As for the suspend example in the book... 'tis true, it is not well
documented. Of course, Brian should have been a little more cautious,
since, as a footnote documents for posterity, the example is Brian's
code, essentially as he wrote it. (And, as Brian just admitted, he
doesn't tend to comment as well as I do. ;-) )


Dave Butenhof 

================================================================
 Q113: Using recursive mutexes and condition variables? 


I have a question regarding recursive mutexes and condition variables.

Given a mutex created with one of the following attributes:

DCE threads
    pthread_mutexattr_setkind_np( &attr, MUTEX_RECURSIVE_NP );

X/Open XSH5 (UNIX98)
   pthread_mutexattr_settype( &attr, PTHREAD_MUTEX_RECURSIVE );

What exactly is the behavior of a pthread_cond_wait() and what effect do
"nested" locks have on this behavior?

Do mixing recursive locks and condition variables make any sense?

This is largely achademic.  However, I (like everyone else in known
space/time) maintain an OO abstraction of a portable subset of Pthreads
and would like to know the appropriate semantics.  

Since the advent of the framework (about 3 years ago) I have managed to
avoid using recursive mutexes.  Unfortunately, my back may be against
the wall on a few new projects and I may be forced to use them.  They
seem to be a real pain.

And yes I promise this concludes my postings on cvs for the forseen
future.


thanks,


--ben

================================================================
 Q114: How to cleanup TSD in Win32? 

>I am forced to use TSD in multithreading existing code. I just ran
>into the problem that, while the destructor function argument to
>pthread_key_create() and thr_keycreate() appears ideal, there is no
>such facility with the NT TlsAlloc() and that related stuff.

It's pretty easy to write the code to provide this facility. Basically
you have to wrap TlsAlloc() and TlsFree() in some additional logic.
This logic maintains a list of all the keys currently allocated for
your process. For each allocated key it stores the address of the
destructor routine for that key (supplied as an argument to the
wrapped version of TlsAlloc()). When a thread exits it iterates
through these records; for every key that references valid data (i.e.
TlsGetValue() returns non-NULL) call the relevant destructor routine,
supplying the address of the thread specific data as an argument.

The tricky part to all this is figuring out when to call the routine
that invokes the destructors. If you have complete control over all
your threads then you can make sure that it happens in the right
place. If, in the other hand, you are writing a DLL and you do not
have control over thread creation/termination this whole approach gets
rather messy. You can do most of the right stuff in the
DLL_THREAD_DETACH section of DllMain() but the thread that attached
your DLL will not take this route, and trying to clean up TLS from
DLL_PROCESS_DETACH is dangerous at best.

Good luck.


Gilbert W. Pilz Jr.
Systems Software Consultant
gwpilz@pobox.com

================================================================
 Q115: Onyx1 architecture has one problem 
Hi there,

I made some parallel measurements on SGI's.

It seemed that the Onyx1 architecture has one problem:
the system bus, which introduces a communication bottleneck.
Memory access and float point calculation introduces traffic
on that bus.

Measurements on the new Onyx2 crossbar-based architecture
suggested that these problems would be solved. However, some early
measurements suggested two thoughts:
1. Float point calculation scales better on the Onyx2 architecture,
which suggests that this problem was really communication related.
(-> crossbar). Going beyond 4 processores (more than one crossbar),
 the scaling goes down.

2. Memory allocation:
Memory allocation (basically a sequential operation) is *really*
slow. Most time is spend at the locking mechanism. This surprises
me, because the pthread mutices I'm using in the code are called
at least as much as the memory allocation, but they are much faster.

Does anybody at SGI has some hints to explain this behaviour?

Thanks,

Dirk
-- 

======== 8< ======= 8< ======= 8< ======= 8< ======= 
Dirk Bartz 					University of Tuebingen

Dirk Bartz  writes:

> 2. Memory allocation:
> Memory allocation (basically a sequential operation) is *really*
> slow. Most time is spend at the locking mechanism.

I have noticed this as well, albeit with the `old' sproc-threads on
Irix-5.3.  ptmalloc seems to be an order of magnitude faster in the
presence of multiple threads on that platform:

  ftp://ftp.dent.med.uni-muenchen.de/pub/wmglo/ptmalloc.tar.gz

However, for Irix-6 with pthreads, you have to use a modified
ptmalloc/thread-m.h file, as I've recently discovered.  I will send
you that file by mail if you're interested; it will also be in the
next ptmalloc release, due out RSN.

Regards,
Wolfram.

================================================================
 Q116: LinuxThreads linked with X11 seg faults. 

Unfortunately the X11 libraries are not compiled with -D_REENTRANT, hence
the problems. You can get the source for the X11 libraries and rebuild them
with the -D_REENTRANT flag and that should help.

If you are using Motif you are out of luck. I spoke to the folks who supply
motif for RedHat Linux. They refused to give me a version recompiled with
the -D_REENTRANT version. They gave me a load of crap about having to test
it and so forth.

I tried using LessTif, but it seemed to be missing too much.

Neil

================================================================
 Q117: Comments about Linux and Threads and X11 

> LinuxThreads linked with X11 by g++ causes calls to the X11 library to seg
> fault.

You can either use Proven's pthread package or LinuxThreads.

Proven's is a giant replacment for the standard libraries that
does user level threads in a single process. 

LinuxThreads uses the Operating system clone() call to implement
threads as seperate processes that share the same memory space.
LinuxThreads seems to be tricky to install as it requires new
versions of the standard libraries in addition to a 2.x kernel and
the pthread library. However, if you get the latest version of RedHat,
you're all set.

I've found Proven's implementation to be much faster, though somewhat
messier to compile and a bit incomplete in it's system call
implementation (remember, it has to provide a substitute for almost
everys system call). Unfortunately I had to switch to LinuxThreads
because the signal handling under Proven's threads was not working
properly.

In particular, disk performance seems to suffer under LinuxThreads.
As far as I can tell, the OS level disk caching scheme gets confused
by all the thread/processes that are created.

It's also a bit unnerving typeing "ps" and seeing fourty copies
of your application running!

...

Unfortunately the X11 libraries are not compiled with -D_REENTRANT, hence
the problems. You can get the source for the X11 libraries and rebuild them
with the -D_REENTRANT flag and that should help.

If you are using Motif you are out of luck. I spoke to the folks who supply
motif for RedHat Linux. They refused to give me a version recompiled with
the -D_REENTRANT version. They gave me a load of crap about having to test
it and so forth.

I tried using LessTif, but it seemed to be missing too much.

Neil



================================================================
 Q118: Memory barriers for synchonization 

Joe Seigh wrote:
> 
> So there are memory barriers in mutexes, contrary to what has been stated
> before in this newsgroup.  Furthermore, it appears from what you are saying is
> that the mutex lock acts as a fetch memory barrier and the mutex unlock
> acts as a store memory barrier, much like Java's mutex definitions.
> Which is not suprising.  Java appears to have carried over quite a bit of the
> POSIX thread semantics.

This is not QUITE correct. First off, the semantic of locking or unlocking a
mutex makes no distinction regarding read or write. In an architecture that
allows reordering reads and writes, neither reads nor writes may be allowed
to migrate beyond the scope of the mutex lock, in either direction. That is,
if the architecture supports both "fetch" and "store" barriers, you must
apply the behavior of both to locking AND unlocking a mutex.

The Alpha, for example, uses MB to prevent reordering of both reads and
writes across the "barrier". There's also a WMB that allows read reordering,
but prevents write reordering. WMB, while tempting and faster, CANNOT be
used to implement (either lock or unlock of) a POSIX mutex, because it
doesn't provide the necessary level of protection against reordering.

Finally, let's be sure we're speaking the same language, ("Gibberishese").
People use "memory barrier" to mean various things. For some, it means a
full cache flush that ensures total main memory coherency with respect to
the invoking processor. That's fine, but it's stronger than required for a
mutex, and it's not what *I* mean. The actual required semantic (and that of
the Alpha) is that a "memory barrier" controls how memory accesses by the
processor may be reordered before reaching main memory. There's no "flush",
nor is such a thing necessary. Instead, you ensure that data (reads and
writes) cannot be reordered past the lock, in either of the processors
involved in some transaction.

An MB preceding the unlock of a mutex guarantees that all data visible to
the unlocking processor is consistent as of the unlock operation. An MB
following the lock of the mutex guarantees that the data visible to the
locking processor is consistent as of the lock operation. Thus, unlocking a
mutex in one thread does not guarantee consistent memory visibility to
another thread that doesn't lock a mutex. Coherent memory visibility, in the
POSIX model, for both readers and writers, is guaranteed only by calling
specific POSIX functions; the most common of which are locking and unlocking
a mutex. A "memory barrier", of any sort, is merely one possible hardware
mechanism to implement the POSIX rules.

/---------------------------[ Dave Butenhof ]--------------------------\

================================================================
 Q119: Recursive mutex debate 

Robert White wrote:
> 
> I STRONGLY DISAGREE with the idea that recursive mutexes "are a bad idea".
> 
> I have made and use a recursive mutex class in several key C++ endeavors.  As a
> low-level tool recursive mutexes are "bad" in that they tend to lead the sloppy
> down dangerous roads.  Conversly, in experienced hands an recursive mutex is a
> tool of simple elegance.  The core thing, as always, is "knowing what you are
> doing".

Hey, look, recursive mutexes aren't illegal, they're not "morally
perverse", and with XSH5 (UNIX98) they're even standard and portable.
So, fine -- if you like them, you use them. Use them as much as you
like, and in any way you like.

But remember that they're ALWAYS more expensive then "normal" mutexes
(unless your normal mutexes are more expensive than they need to be for
the platform!). And remember that WAITING on a condition variable using
a recursively locked mutex simply won't work. So, if you're using a
condition variable to manage your queue states, you need to at least
analyze your lock usage sufficiently to ensure that the wait will work.
And, once you've done that, it's a simple step to dropping back to a
normal mutex.

There are definitely cases where the expense is acceptable, especially
when modifying existing code -- for example, to create a thread-safe
stdio package. The performance isn't "extremely critical", and you don't
need to worry about condition wait deadlocks (there's no reason to use
them in stdio). Sorting out all of the interactions between the parts of
the package is difficult, and requires a lot of new coding and
reorganization -- and implementing some of the correct semantics gets
really tricky.

Don't waste time optimizing code that's not on the critical path. If
you've got code that's on your critical path, and uses recursive
mutexes, then it's NOT optimized. If you care, you should remove the
recursive mutexes. If you don't care, fine. If the use of recursive
mutexes in non-critical-path code doesn't put it on the critical path,
there's no reason to worry about them.

Still, I, personally, would use a recursive mutex in new code only with
extreme reluctance and substantial consideration of the alternatives.

/---------------------------[ Dave Butenhof ]--------------------------\

[I echo Dave's "extreme reluctance and substantial consideration of the 
 alternatives" -Bil]
================================================================
 Q120: Calling fork() from a thread 

> Can I fork from within a thread ?

Absolutely.

> If that is not explicitly forbidden, then what happens to the other threads in
> the child process ?

There ARE no other threads in the child process. Just the one that
forked. If your application/library has background threads that need to
exist in a forked child, then you should set up an "atfork" child
handler (by calling pthread_atfork) to recreate them. And if you use
mutexes, and want your application/library to be "fork safe" at all, you
also need to supply an atfork handler set to pre-lock all your mutexes
in the parent, then release them in the parent and child handlers.
Otherwise, ANOTHER thread might have a mutex locked when one thread
forks -- and because the owning thread doesn't exist in the child, the
mutex could never be released. (And, worse, whatever data is protected
by the mutex is in an unknown and inconsistent state.)

One draft of the POSIX standard had included the UI thread notion of
"forkall", where all threads were replicated in the child process. Some
consider this model preferable. Unfortunately, there are a lot of
problems with that, too, and they're harder to manage, because there's
no reasonable way for the threads to know that they've been cloned. (UI
threads allows that blocking kernel functions MAY fail with EINTR in the
child... but that's not a very good basis for a recovery mechanism.)
After much discussion and gnashing of teeth and tearing of hair, the
following draft removed the option of forkall.

> Is there a restriction saying that it's OK provided the child immediately does
> an exec ?

Actually, this is the ONLY way it's really safe, unless every "facility"
in the process has proper and correct forkall handling to protect all of
the process state across the fork.

In fact, despite the addition of forkall handlers in POSIX 1003.1c, the
standard specifically says that the child process is allowed to call
only async signal safe functions prior to exec. So, while the only real
purpose of forkall is to protect the user-mode state of the process,
you're really not guaranteed that you can make any use of that state in
the child.

> What if I do this on a multiprocessor machine ?

No real difference. You're more likely to have "stranded" mutexes and
predicates, of course, in a non-fork-safe process that forks, becuase
other threads were doing things simultaneously. But given timeslicing
and preemption and other factors, you can have "other threads" with
locked mutexes and inconsistent predicates even on a uniprocessor.

Just remember, that, in a threaded process, it's not polite to say "fork
you" ;-)

/---------------------------[ Dave Butenhof ]--------------------------\

================================================================
 Q121: Behavior of [pthread_yield()] sched_yield() 

> > I have a question regarding POSIX threads on Linux and Solaris. The
> > program below compiles and links well on both systems, but instead of the
> > expected "100000, " it always prints out
> > "100000, 0", so the thread is not really ever started.
> 
> 
> 
> Well, both sets of output are legal and correct for the code you supplied.

Yes, this is correct.

> First, you see [p]thread_yeild does not say "give control to another thread"
> it says "if there is another thread that can be run, now might be a good time
> do do that".  The library is under no obligation to actually yeild.  (there
> is a good explaination of this elsewhere in this group, but it has to do with
> the fact that you are running under SCHED_OTHER semantics which are
> completely unspecified semantics, go figure.)

Just for clarity...

	pthread_yield is an artifact of the obsolete and crufty old
	DCE thread implementation (loose interpretation of the 1990
	draft 4 of the POSIX thread standard). It doesn't exist in
	POSIX threads.

	thr_yield is an artifact of the UI threads interface, which
	is, (effectively though not truly), Solaris proprietary.

	sched_yield is the equivalent POSIX function.

As Robert said, POSIX assigns no particular semantics to the SCHED_OTHER
scheduling policy. It's just a convenient name. In the lexicon that we
developed during the course of developing the realtime and thread POSIX
standards, it is "a portable way to be nonportable". When you use
SCHED_OTHER, which is the default scheduling policy, all bets are off.
POSIX says nothing about the scheduling behavior of the thread.
(Although it does require a conforming implementation to DOCUMENT what
the behavior will be.)

Because there's no definition of the behavior of SCHED_OTHER, it would
be rather hard to provide any guarantees about the operation of the
sched_yield function, wouldn't it?

If you want portable and guaranteed POSIX scheduling, you must use the
SCHED_FIFO or SCHED_RR scheduling policies (exclusively). And, of
course, you need to run on a system that supports them.

> Next, the number of threads in a (POSIX) program does not necessarily say
> anthing about the number of actual lightweight processes that will be used to
> execute the program.  In your example there is nothing that "forcably" causes
> the main thread to give up the processor (you are 100% CPU related) so your
> first thread runs through to completion.  An identically arranged ADA program
> (which wouldn't quite be possible 8-) would have equally unstable results.
> (I've seen students write essentially this exact program to "play with" tasks
> and threads in ADA and C, but the program is not valid in any predictable
> way.)

POSIX doesn't even say that there's any such thing as a "light weight
process". It refers only obliquely to the hypothetical concept of a
"kernel execution entity", which might be used as one possible
implementation mechanism for Process Contention Scope thread scheduling.

> Finally, POSIX only says that there will be "enough" LWPs at any moment to
> ensure that the program as a whole "continues to make progress".

That's not strictly true. All POSIX says is that a thread that blocks
must not indefinitely prevent other threads from making progress. It
says nothing about LWPs, nor places any requirements upon how many there
must be.

> When you do the SIGINT from the keyboard you are essentially causing the
> "current" thread to do a pthread_exit/abort. Now there is only one thread
> left, the "second" one, so to keep the program progressing that one get's the
> LWP from the main thread.  That is why you see the second start up when you
> do a "^C"...

SIGINT shouldn't "do" anything to a thread, on a POSIX thread system. IF
it is not handled by a sigaction or a sigwait somewhere in the process,
the default signal action will be to terminate the process (NOT the
thread).

It's not clear from the original posting exactly where the described
results were seen: Linux or Solaris? My guess is that this is Linux,
with the LinuxThreads package. Your threads are really cloned PROCESSES,
and I believe that LinuxThreads still does nothing to properly implement
the POSIX signal model among the threads that compose the "process".
That may mean that, under some circumstances, (and in contradiction to
the POSIX standard), a signal may affect only one thread in the process.
The LinuxThreads FaQ says that SIGSTOP/SIGCONT will affect only the
targeted thread, for example. Although it also says that threads "dying"
of a signal will replicate the signal to the other threads, that might
not apply to SIGINT, or there might be a timing window or an outright
hole where that's not happening in this case.

LinuxThreads is, after all, a freeware thread package that's from all
reports done an excellent job of attacking a fairly ambitious goal. A
few restrictions and nonconformancies are inevitable and apparently
acceptable to those who use it (although it's gotta be a portability
nightmare for those who use signals a lot, you're always best off
avoiding signals in threaded programs anyway -- a little extra
"incentive" isn't a bad thing). If you see this behavior on Solaris,
however, it's a serious BUG that you should report to Sun.

> The very same program with a single valid "operational yeild" (say reading a
> character from the input device right after the pthread_create()) will run at
> 100% CPU forever because it will never switch *OUT* of the second thread.

At least, that's true on Solaris, where user threads aren't timesliced.
To get multiple threads to operate concurrently, you need to either
manually create additional LWPs (thr_setconcurrency), or create the
threads using system contention scope (pthread_attr_setscope) so that
each has its own dedicated LWP. Solaris will timeslice the LWPs so that
multiple compute-bound threads/processes can share a single processor.
LinuxThreads directly maps each "POSIX thread" to a "kernel thread"
(cloned process), and should NOT suffer from the same problem. The
kernel will timeslice the "POSIX threads" just as it timeslices all
other processes in the system. On Digital UNIX, the 2-level scheduler
timeslices the user ("process contention scope") threads, so, if a
compute-bound SCHED_OTHER thread runs for its full quantum, another
thread will be given a chance to run.

> In essence there is no good "Hello World" program for (POSIX) threads (Which
> is essentially what you must have been trying to write 8-).  If the threads
> don't interact with the real world, or at least eachother, the overall
> program will not really run.  The spec is written to be very responsive to
> real-world demands.  That responsiveness in the spec has this example as a
> clear degenerate case.

That's not true. "Hello world" is easy. If the thread just printed
"Hello world" and exited, and main either joined with it, or called
pthread_exit to terminate without trashing the process, you'd see
exactly the output you ought to expect, on any conforming POSIX
implementation. The problem is that the program in question is trying to
execute two compute-bound threads concurrently in SCHED_OTHER policy:
and the behavior of that case is simply "out of scope" for the standard.
The translation of which is that there's no reasonable assumption of a
portable behavior.

/---------------------------[ Dave Butenhof ]--------------------------\


================================================================
 Q122: Behavior of pthread_setspecific() 

> Can you explain the discrepancy between your suggestion and the
> following warning, which I found in the SunOS 5.5.1 man page for
> "pthread_setspecific".
> 
> ******************************************************************
> WARNINGS
>      pthread_setspecific(),                pthread_getspecific(),
>      thr_setspecific(),  and  thr_getspecific(),  may  be  called
>      either explicitly, or implicitly from a thread-specific data
>      destructor function.  However, calling pthread_setspecific()
>      or thr_setspecific() from a destructor may  result  in  lost
>      storage or infinite loops.
> 
> SunOS 5.5.1         Last change: 30 Jun 1995                    4
> ******************************************************************
> 
> I'm not sure how an infinite loop might occur, while using
> "pthread_setspecific" in a destructor.  Do you know the answer?

We're talking about two different things.

1) What the standard says, which is that the destructor is called, and
   may be called repeatedly (until a fixed, implementation specified
   limit, or forever), until the thread-specific data values for the
   thread become NULL. Because the standard doesn't say that the
   implementation is required to clear the value for each key as the
   destructor is called, that requirement is, implicitly, placed on the
   application. (This oversight will be corrected in a future update
   to the standard.)

   In order to set the value to NULL, you clearly must call the function
   pthread_setspecific() within the destructor. Note that setting the
   value to NULL within the destructor will work either with the current
   standard (and the current LinuxThreads literal implementation) AND
   with the fixed standard (and most other implementations, which have
   already implemented the correct semantics, figuring that an infinite
   loop usually is not desirable behavior).

2) The correct POSIX semantics, which are implemented by Solaris and
   Digital UNIX. (Probably also by IRIX, HP-UX, and AIX, although I
   haven't been able to verify that.) The Solaris manpage warning is
   imprecise, however. There's no problem with a destructor explicitly
   setting the value to NULL. The warning SHOULD say that setting a
   thread-specific data value to any non-NULL value within a destructor
   could lead to an infinite loop. Or, alternately, to a memory leak, if
   the new value represents allocated heap storage, and the system has
   a limit to the number of times it will retry thread-specific data
   destruction.

/---------------------------[ Dave Butenhof ]--------------------------\

 Q123: Linking under OSF1 3.2: flags and library order  

Joerg Faschingbauer wrote:

> Hi,
>
> recently I posted a question about the correct linking order under
> Solaris 2.4. Got some valuable hints, thanks.
>
> I have a similar problem now, this time under OSF1 3.2. Can anybody
> tell me if the following is correct? I could not find any hints on
> that topic in the man pages.
>
> gcc ... -ldnet_stub -lm -lpthreads -lc_r -lmach
>
> Does pthreads need stuff from c_r, or the other way around? Do I need
> mach at all? Do I need dnet_stub at all?

In a threaded program prior to Digital UNIX 4.0, EVERYTHING needs
libc_r, because libc is not thread-safe. Yes, the thread library
requires libmach, and, because of bizarre symbol preemption requirements
(which, for trivia junkies, were at one time required by OSF for "OSF/1"
branding), if you don't include libmach explicitly things might not work
out right. You must specify libmach BEFORE libc_r. You don't need
-ldnet_stub unless YOU need it (or some other library you're including).
We certainly don't use it.

The best way to build a threaded program on 3.2 is to use "cc -threads".
If you're going to use gcc, or an older cxx that doesn't support
"-threads", or if you need to use ld to link, then the proper expansion
of "-threads" is:

     for compilation:
          -D_REENTRANT
     for linkage:
          -lpthreads -lmach -lc_r

The linkage switches must be the LAST libraries, exclusive of libc. That
is, if you were using ld to link, ...

     ld <.o files...> -lpthread -lmach -lc_r -lc crt0.o

I don't believe the position of -lm with respect to the thread libraries
will matter much, since it's pretty much independent. If you use -lm
-threads, however, libm will precede the thread libraries, and that's a
good standard to follow.

A side effect of "-threads" is that ld will automatically look for a
reentrant variant of any library that you specify. That is, if you
specify "-lfoo", and there's a "libfoo_r", ld will automatically use
libfoo_r. If you don't use -threads, you'll need to check /usr/shlib (or
/usr/lib if you're building non-shared) for reentrant variants.

Note that, to compile a DCE thread (draft 4) threaded program once you
move to Digital UNIX 4.0 or higher, the compilation expansion of
-threads will need to be changed to "-D_REENTRANT -D_PTHREAD_USE_D4",
and the list of libraries should be "-lpthreads -lpthread -lmach -lexc".
There's no libc_r on 4.0 (libc is fully thread-safe), and you need
libexe since we've integrated with the standard O/S exception mechanism.
Note the distinction between libpthread (the "core" library implementing
POSIX threads), and libpthreads (the "legacy" library containing DCE
thread and CMA wrapper functions on top of POSIX thread functions).

Minor additional notes: as of Digital UNIX 4.0D we've dropped the final
dependencies on the mach interfaces, so libmach is no longer required
(you'll get smaller binaries and faster activation by omitting it once
you no longer need to support earlier versions). And, of course, once
you've moved to 4.0 or later, you should port to POSIX threads, in which
case you can drop -lpthreads and -D_PTHREAD_USE_D4.

/---------------------------[ Dave Butenhof ]--------------------------\
================================================================
 Q124: What is the TID during initialization?  

Lee Sailer wrote:

> In a program I am "maintaining", there is a
>
>    foo = RWThreadID();
>
> call at global scope.  Conceptually, this gets called before main().
> Does this seem OK?  Can this code rely on the "thread" being used to do
> initial construction to be the same as the "main thread"?

[For example, the .init sections of libraries run before main() starts. -Bil]

While the assumption will likely be true, most of the time, it strikes me
as an extremely dangerous and pointless assumption. There are a lot of
reasons why it might NOT be true, sometimes, on some platforms, under some
circumstances. There's no standard or rule of ettiquette forbidding a
difference. Even if the "thread" is the same, the "thread ID" might change
as things get initialized.

I recommend avoiding any such assumptions.

/---------------------------[ Dave Butenhof ]--------------------------\

================================================================
 Q125: TSD destructors run at exit time... and if it crashes?  

Sebastien Marc wrote:

> On Solaris you can associate a function (called destructor) that will be
> called at the termination of the thread, even if it crashes.

Almost. Both POSIX and UI threads interfaces include thread-specific data.
When you create a thread-specific data (TSD) key, you can specify a
destructor function that will be run when any thread with a non-NULL value
for that key terminates due to cancellation [POSIX only] or voluntary thread
exit (return from the thread's start routine, or a thread exit call --
pthread_exit or thr_exit).

Yes, you can use that as a sort of "atexit" for threads, if you make sure
that each thread uses pthread_setspecific/thr_setspecific to SET a non-NULL
value for the TSD key. (The default value is NULL, and only the thread itself
can set a value.)

However, that doesn't help. There is simply no way that a thread can "crash"
without taking the process with it. A unhandled signal will never terminate a
thread -- either the signal is ignored, or it does something to the process
(stop, continue, terminate). TSD destructors are NOT run:

   * on the child side of a fork
   * in a call to exec
   * in process termination, regardless of whether that termination is
     voluntary (e.g., a call to exit) or involuntary (an unhandled signal).

In all those cases, threads quietly "evaporate", leaving no trace of their
existence. No TSD destructors, no cleanup handlers, nothing. Gone. Poof.

/---------------------------[ Dave Butenhof ]--------------------------\

================================================================
 Q126: Cancellation and condition variables  

Marcel Bastiaans wrote:

> Anyone:
>
> I appear to be missing something in my understanding of how condition
> variables work.  I am trying to write a multithreaded program which is
> portable to various platforms.  I am unable to cancel a thread if it is
> waiting on a condition variable which another thread is waiting on also.
> The problem can easily be reproduced on both Solaris 2.5 and HP-UX 10.10.  A
> simple program which demonstrates my problem is shown below.  This program
> sample uses the HP-UX pthreads library but the problem also appears when
> using Solaris threads on Solaris 2.5.

In any case... yes, you are missing something. The program, as written, will
hang on any conforming (or even reasonably correct) implementation of either
DCE threads or POSIX threads. (To put it another way, any implementation on
which it succeeds is completely broken.)

> Is there a problem in this program which I don't understand?  I cannot use
> cleanup handlers because not all platforms support them.  Any help would be
> greatly appreciated.

If you can use cancellation, you can use cleanup handlers. Both are part of
both DCE threads (what you're using on HP-UX 10.10) and POSIX threads (what you
probably are, and, at least, should be, using on Solaris 2.5.) If you've got
cancellation, and you don't have cleanup handlers, you've got an awesomely
broken implementation and you should immediately chuck it.

When you wait on a condition variable, and the thread may be cancelled, you
MUST use a cleanup handler. The thread will wake from the condition wait with
the associated mutex locked -- even if it was cancelled. If the thread doesn't
then unlock the mutex before terminating, that mutex cannot be used again by
the program... it will remain locked by the cancelled thread.

> #include 
> #include 
> #include 
>
> pthread_cond_t cond;
> pthread_mutex_t mutex;
>
> void * func(void *)
> {
>    // Allow this thread to be cancelled at any time
>    pthread_setcancel(CANCEL_ON);
>    pthread_setasynccancel(CANCEL_ON);

Serious, SERIOUS bug alert!! DELETE the preceding line before proceeding with
this or any other program. Never, ever, enable async cancelation except on
small sections of straight-line code that does not make any external calls.
Better yet, never use async cancel at all.

In any case, you absolutely CANNOT call any POSIX (or DCE) thread function with
async cancellation enabled except the ones that DISABLE async cancel. (For
bizarre and absolutely unjustifiable reasons [because they're wrong], POSIX
threads also allows you to call pthread_cancel -- but don't do it!)

>    // Wait forever on the condition var
>    pthread_mutex_lock(&mutex);
>    for(;;) {
>       pthread_cond_wait(&cond, &mutex);
>    }
>    pthread_mutex_unlock(&mutex);
>    return 0;
> }

I suspect your problem is in cancelling the second thread. As I said,
cancellation terminates the condition wait with the associated mutex locked.
You're just letting the thread terminate with the mutex still locked. That
means, cancelled or not, the second thread can never awaken from the condition
wait. (At a lower level, you could say that it HAS awakened from the condition
wait, but is now waiting on the mutex... and a mutex wait isn't cancellable.)

The answer is... if you use cancellation, you must also use cleanup handlers.
(Or other, non-portable equivalent mechanisms, such as exception handlers or
C++ object destructors... on platforms where they're implemented to
interoperate with cancellation. [Both Solaris and Digital UNIX, for example,
run C++ destructors on cancellation.])

/---------------------------[ Dave Butenhof ]--------------------------\
================================================================
 Q127: RedHat 4.2 and LinuxThreads?  

> > The Linux kernel has supported multithreading for a very long time.
> 
> thank you for the info, Bill.  the man page for clone that ships with
> Red Hat 4.2 states that clone does not work.  here are my questions.
> they all relate to Red Hat 4.2:
> 
> 1. does clone work for all defined parameter values?
> 2. where can i find a list of the c library api's that are not
> reentrant?
> 3. does RedHat 4.2 install LinuxThreads if "everything" is selected?
> 
> 
> > Until recently, the API definition for POSIX thread support was
> > contained in the LinuxThreads package, but that's just a wrapper
> > around the kernel's built=in functioning.  With the releace of libc6
> > (GNU libc) the LinuxThreads functionality is more tightly integrated
> > into the basic C library,
> 
> do you mean that the POSIX thread api's are now in libc so that
> LinuxThreads is obsolete?

With the glibc2 (2.0.5 c is current I think) LinuxThreads is
obsolete. However, you have to get yourself the additional
glibc-linuxthreads package, but that's detail.

AFAIK glibc2 is still in the beta stadium, but it works quite
well. Moreover, it is recommended to use glibc2 for multithreading
rather than libc5. As H.J.Lu, libc5's maintainer, once stated: "I'm
surprised it works at all" (or so).

You can install a "beta" of the libc6 (aka glibc2) as a secondary C
library against which you link your program, and keep the good old
libc5 as the primary library which the system related programs use.

Take a look at 

http://www.imaxx.net/~thrytis/glibc/

for HOWTOs etc.

Joerg
----------------------------------------------------------------------------
Joerg Faschingbauer                                     jfasch@hyperwave.com
Voice: ++43/316/820918-31                            Fax: ++43/316/820918-99
----------------------------------------------------------------------------

================================================================
 Q128: How do I measure thread timings?  
Andy Sunny wrote:

> I'm conducting some research to measure the following things about
> pthreads using a Multikron II Hardware Instrumentation Board from NIST
> 1) thread creation time (time to put thread on queue)
> 2) thread waiting time (time that thread waits on queue)
> 3) thread execution time (time that thread actually executes)
>
> Are there any decent papers that explain the pthreads run time system
> and scheduling policy in DETAIL? I have read Frank Mueller's (FSU) paper
> and am trying to obtain the standard from IEEE. What is the latest
> version of the standard and will it help me find the proper libraries
> and functions need to measure the above items?

The standard is unlikely to be of any help to you. It says nothing at all
about implementation. POSIX specifies SOURCE-LEVEL interfaces, and
describes the required portable semantics of those interfaces.
Implementation details are (deliberately, properly, and necessarily) left
entirely to the creator of each implementation. For example, there's no
mention of libraries -- an embedded system, for example, might include all
interfaces in an integrated kernel; and that's fine.

What you need is a document describing the internal implementation details
of the particular system you're using. If the vendor can't supply that,
you'll need to create it yourself -- either by reading source, if you can
get it, or by flailing around blindly in the dark and charting the walls
you hit.

/---------------------------[ Dave Butenhof ]--------------------------\
================================================================
 Q129: Contrasting Win32 and POSIX thread designs  
Arun Sharma wrote:

> On Mon, 24 Nov 1997 18:10:13 GMT, Christophe Beauregard wrote:
>
>         c>  while thread context is a Windows concept.
>
> How so ? pthreads don't have contexts ?

This looks like an interesting discussion, of which I've missed the
beginning. (Perhaps only the followup was cross-posted to
comp.programming.threads?) Anyway, some comments:Anything has "context".
A thread is an abstraction of the executable state traditionally
attributed to a process. The "process" retains the non-executable state,
including files and address space. Why would anyone contend that "thread
context is a Windows concept"? I can't imagine. Maybe it's buried in the
unquoted portion of the orignal message. And then again, some people
think Microsoft invented the world.

>         c> Generally, you'll find that pthreads gives you less control
>         c> over how a thread runs.  There are very good reasons for
>         c> this (one being portability, another being safety).
>
> In other words, it has to be the least common denominator in the
> fragmented UNIX world. No wonder people love NT and Win32 threads.

POSIX threads gives you far more real control over threads than Win32
(for example, far superior realtime scheduling control). What it doesn't
give you is suspend/resume and uncontrolled termination. Those aren't
"control over how a thread runs". They are extraordinarily poor
programming mechanisms that can almost never be used correctly. Yes, to
some people the key is "almost never", and one may argue that they should
be provided anyway for that 0.001% of applications that "need" it. (But
those of us who actually support threaded interfaces might also point out
that these extremely dangerous functions are for some reason particularly
tempting to beginners who don't know what they're doing -- resulting in
very high maintenance costs, which mostly involves helping them debug
problems in their code.)

This isn't an example of "fragmented UNIX" -- it's UNIX unity, with a
wide variety of different "UNIX camps" reaching a concensus on what's
necessary and useful.

While the Win32 interface comprises whatever the heck a few designers
felt like tossing in, POSIX was carefully designed and reviewed by a
large number of people, many of whom knew what they were doing. Omitting
these functions was a carefully considered, extensively discussed, and
quite deliberate decision. The Aspen committee that designed the thread
extensions to POSIX for the Single UNIX Specification, Version 2,
proposed suspend/resume -- they were later retracted by the original
proposer (with no objections). A POSIX draft standard currently under
development, 1003.1j, had proposed a mechanism for uncontrolled
termination, with the explicit recognition that it could be used (and
then only with extreme care) only in carefully constructed embedded
systems. It, too, was later withdrawn as the complications became more
obvious. (The notion that you can regain control of a process when you've
lost control of any one thread in the process is faulty, because all
threads depend completely on shared resources. If you've lost control of
a thread, you don't know the state of the process -- how can you expect
it to continue?)

>         c> Basically, using signals for dealing with threads is a Bad
>         c> Thing and people who try generally get screwed.
>
> It doesn't have to be so. That's an implementation problem.

Yes, it does have to be so, because signals are a bad idea to begin with.
Although there were enormous complications even before threads, the
concept becomes all but unsupportable with the addition of full
asynchronous execution contexts to the traditional process.

The "synchronous" signals, including SIGSEGV, should be language
exceptions. The other "asynchronous" signals should be handled
synchronously in independent contexts (threads). If you think about it,
that's what signals were attempting to do; the condition exists as a
separate execution context (the signal handler). Unfortunately, a signal
preempts the hardware context of the main execution context,
asynchronously. That's a really, really bad idea. Although people have
always casually done things like calling printf in signal handlers, too
few people realize that's always been incorrect and dangerous -- only a
small list of UNIX functions are "async-signal safe". The addition of
threads, however, allowing the process to have multiple contexts at any
time, increases the chances that some thread will be doing something that
will conflict with improper use of non async-signal safe functions at
signal level.

> Portability doesn't necessarily have to cripple the API.

And, in fact, it doesn't. It results in a well-designed and robust
interface that can be efficiently implemented everywhere. I'm not arguing
that the POSIX interface is perfect. There is room for additions, and the
Single UNIX Specification, Version 2, makes a good start. Other areas to
consider for future standardization would include debugging and analysis
interfaces. There are POSIX standards in progress to improve support for
"hard realtime" environments (for example, putting timeouts on all
blocking functions to control latency and help diagnose failures).

/---------------------------[ Dave Butenhof ]--------------------------\
================================================================
 Q130: What does POSIX say about putting stubs in libc?  

Patrick TJ McPhee wrote:

> I'd like to know what Posix has to say about putting stubs in libc.
> Is it permitted? Is it required? What return code can we expect to
> receive from such a stub, and how can we portably ignore it?

POSIX doesn't have ANYTHING to say. POSIX 1003.1 doesn't really recognize the
existance of the concept of a "library". It defines a set of SOURCE LEVEL
interfaces that shall be provided by implementations and that may be used by
applications to achieve certain portable semantics. Now, 1003.2 says a little
about libraries. That is, there's something with a ".a" file suffix, and
there's a utility called "ar" to create them, and a utility called "c89" with
a "-l" switch that may read from a "lib.a" (and may also read
additional file suffixes, e.g., .so). 1003.2 doesn't say anything about which
symbols may or should be resolved from which libraries, though, and hasn't
been updated since 1003.1c-1995 (threads), in any case.

So, if your system's  provides a definition for _POSIX_THREADS, then
1003.1c-1995 tells you that you can potentially call pthread_create. It does
not tell you which libraries you need to link against. UNIX98 does specify
that, for c89, the proper incantation is "-lpthread". But even that's not the
same as a requirement that the symbols must resolve from, and only from, a
libpthread library: only that you're not allowed to build a portable threaded
application without REQUESTING libpthread. (And if you use cc instead of c89,
UNIX98 doesn't help you any more than POSIX, aside from the gentle SUGGESTION
that an implementation provide the thread implementation in a libpthread --
which had, in any case, already become the defacto industry standard.)

So, yes, it's "permitted", and, no, it's not "required".

If you're building an APPLICATION using threads, there's no confusion or
problem. You build according to the rules of the platform, and you've got
threads, wheresoever they might reside. If you try to use threads without
building properly, all bets are off, because you blew it. If you're getting
the interfaces accidentally from somewhere else, that's nobody's fault but
your own.

If you're trying to build thread-safe code that doesn't use threads, you've
got a portability problem. No standard will help you accomplish this. That's
too bad. Requiring libc "stubs" would be one way out -- but as I've already
said, (and as I'll reiterate in the next paragraph!), the Solaris
implementation has some serious limitations of which I don't approve. I would
not consider that an acceptable standard. I'm not entirely happy with our own
solution (a separate set of "tis" interfaces), either, because extra
interfaces are nobody's friend. One might say that there is room here for
innovation. ;-)

If you're trying to build a library that uses threads, regardless of whether
the main program uses threads -- well, you're in trouble again. You SHOULD be
able to simply build it as if you were building a threaded application, and it
should work. Unfortunately it won't work (portably) unless the main program is
linked against the thread library(s), whether or not it needs them. Symbol
preemption will work against you if there are "stubs" for any functions in a
library that will be searched by direct dependencies of the main program.
(Even if your library is searched first, ITS direct dependencies will go at
the end of the list.) That's the problem with the Solaris libc stubs. (I'd
like to say that Digital UNIX avoids this, and that's certainly the intent;
but unfortunately it's not yet entirely true. Although there are no stubs
conflicting with libpthread, we depend on the libexc exception library, which
has a conflicting stub in libc. Luckily, this affects relatively few
operations -- but, technically, it still means it doesn't work.)

On the other hand, your final question is easy. There's no need to "portably
ignore" the errors that a stub might generate. Look, if you try to create a
thread, it either succeeds or it fails. You get back 0, and it worked.
Anything else, and it failed. If the failure is EAGAIN, you might choose to
try again later. Otherwise... hey, you're just not going to create that
thread, so deal with it. The only question is: can you live with that? If you
don't NEED to create a thread, go on with life, single threaded. If you NEED
to create the thread, then you're done. (Whether you return a failure to your
caller, or abort the process, probably depends on what you're trying to do,
and how your interface is designed.) It really doesn't matter whether you got
activated against libc stubs or a real thread library that for some reason
refuses to create the thread. You're not going to do what you wanted to do,
and that's that.

/---------------------------[ Dave Butenhof ]--------------------------\
================================================================
 Q131: MT GC Issues  

[See Geodesic Systems (www.geodesic.com)			-Bil]

Sanjay Ghemawat wrote:

> All collectors I have known of and read about control both the allocation
> and the deallocation of objects.  So it is fairly easy for them to grab
> all of the locks required before suspending threads.  The only problem
> here might be locks held within the operating system on behalf of a thread
> that is about to be suspended.  Even here, if one is using a thread interface
> like the one provided by Mach,  a call to "thread_abort" will pop the
> thread out of the OS.

There is no general or portable mechanism equivalent to
thread_abort, and it is pretty limited even on Mach. (First,
you have to know it's in a blocking Mach call.)

> >Furthermore, suspend/resume may possibly be necessary for concurrent garbage
> >collection (I haven't yet been convinced of that -- but I haven't found a good
> >alternative, either), but it's definitely far from desirable. It's an ugly and
> >stupendously inefficient kludge. Remember, you have COMPLETELY STOPPED the
> >application while garbage collecting. That's GOOD? Parallel applications want
>
> First of all, most incremental/concurrent collectors only stop the
> application while they find all of the pointers sitting in thread
> stacks/registers/etc.  The collectors that provide better real-time
> guarantees tend to make other operations (such as storing a pointer
> onto a stack) expensive.  I think there are two classes of systems
> here: hard real-time and others.  A good performance tradeoff for
> systems that do not require hard real-time bounds is to use an
> incremental/concurrent collector that may introduce pauses, but does
> not slow down the mutator with lots of book-keeping work.  So I think
> the argument that suspend/resume are bad only applies to some systems,
> not all.  Probably not even the vast majority of systems that run on
> desktops and commercial servers.

Look, if you want concurrency, you DON'T want pauses. I
already acknowledged that there might not be an alternative,
and that we certainly don't know of any now.  Maybe it's the
best tradeoff we can get. Stopping all the threads is still
bad.  Necessity does not equate to desirability. If you
really think that it is BENEFICIAL to stop the execution of
a concurrent process, argue on. Otherwise, drop it.

> >application while garbage collecting. That's GOOD? Parallel applications want
> >concurrency... not to be stopped dead at various unpredictable intervals so
> >the maintenance staff can check the trash cans. There has gotta be a better
> >way.
>
> So we should all wait and not use garbage-collection in multi-threaded
> programs until that better way is found?  I don't see why you are so
> vehemently set against suspend/resume.  It solves real problems for
> people implementing garbage collection.  Yes there are tradeoffs
> here: suspend/resume have their downside, but that doesn't mean we
> should ignore them.

Because suspend and resume are poor interfaces into the
wrong part of a scheduler for external use. They are
currently used (more or less) effectively for a small class
of specialized applications (e.g., garbage collection). They
are absolutely inappropriate for general use. The fact that
a bad function "can be used effectively" doesn't mean it
should be standardized. Standards do not, and should not,
attempt to solve all possible problems.

> So do that.  I don't think there are an unbounded number of such
> required operations.  In fact I have implemented a collector for a
> multi-threaded environment that requires just three such operations
> suspend, resume, and reading registers from a suspended thread.
> And folding in the register-state extraction into the suspend call
> seems like a fine idea.

Now, are we talking about "garbage collectors", or are we
talking about suspend and resume? All this rationalization
about garbage collection spills over naturally into a
discussion of suspend and resume -- but that's happenstance.
Sure, our concurrent GC systems, and especially Java, use
suspend/resume. But that's "because it was there", and
solved the problem of pinning down threads long enough to
get their state. But the function required of concurrent
garbage collection is not "suspend all threads and collect
their registers, then resume them". The required function is
"acquire a consistent set of live data root pointers within
the process".

Yes, there are a bounded set of operations required for GC
-- and that has nothing at all to do with suspend or
resume. If the argument for standardizing suspend and resume
is to revolve entirely around the needs of today's
semi-concurrent GC, then we should be designing an interface
to support what GC really needs, not standardizing an ugly
and dangerous overly-generalized scheduling function that
can be (mis-)used to implement one small part of what GC
needs.

> >> Since the implementation basically needs to be there for any platform
> >> that supports Java, why not standardize it?  Alternatively, if there is
> >> a really portable solution using signals, I would like to see it
> >> advertised.
> >
> >Any "why not" can be rephrased as a "why". Or a "so what".
>
> Oh come on.  By this argument, you could do away with all standards.
> The reason for standards is so that multiple implementations with
> the same interface can exist and be used without changing the
> clients of the interface.

And by the converse, perhaps we should standardize every
whim that comes into anyone's head? Baloney. We should
standardize interfaces that are "necessary and
sufficient". Determining exactly of what that consists is
not always easy -- but it's important because standards have
far- and long-reaching consequences.

> I apologize if this message has come across as a bit of a rant, but
> I am tired of people assuming that everyone who asks for suspend/resume
> must be an idiot who does not understand the available thread
> synchronization mechanisms.  There are legitimate uses for
> suspend/resume, of course with some performance tradeoffs.  By
> making the decision to not implement them in a thread library,
> you are taking away the ability of the clients of the library
> to decide on the tradeoff according to their need.  That makes
> the thread library less useful.

I guess we're even -- because I'm tired of hearing people
insist that because they want suspend/resume, it must be
universally accepted as "a cool thing" and forced down
everyone's throat. It's not a cool thing. And, by the way, I
have yet to hear of a single truly legitimizing use. The use
of suspend and resume by GC is an expedient hack. It isn't
really accomplishing what GC needs. It's far more
heavyweight than required, (as you pointed out, a GC system
suspends threads to get a consistent set of root pointers,
NOT because it wants to suspend the threads), and it doesn't
provide half the required capabilities (after all, the real
goal is to get the pointers -- the registers).

As for your final dig, I'm tempted to laugh. You know what
makes a thread library less useful? Providing unsupportable
functions that are nearly impossible to use safely and that
therefore result in significant support costs, preventing
the development team from doing work that would provide
useful features and fixing real problems.

/---------------------------[ Dave Butenhof ]--------------------------\

================================================================
 Q132: Some details on using CMA threads on Digital UNIX  

bmoss@nsg.dowjones.com wrote:

> I'm trying to port code from and HP that used the cma threads package to
> a DEC Alpha with the posix package. I've found that some of the standard
> header files (DEC C++) have conflicting definitions (e.g., sys/types.h
> and pthreads_exc.h). Has anyone encountered this porblem and is there
> some simple conversion utility or a better library to use is such a port.

A number of questions present themselves immediately, including:

  1. What version of Digital UNIX are you using?
  2. Are you trying to compile with CMA, DCE thread (draft 4 POSIX), or true
     POSIX?
  3. What compiler/link options are you specifying?
  4. What headers do you include?
  5. What is actually happening?

A few comments:

  1. Digital UNIX provides both DCE thread (draft 4 POSIX, both "standard"
     and exception-raising) and CMA interfaces, as in any DCE
     implementation. To use these, compile with "cc -threads" or "cxx
     -threads", and link using the same. If you can't, compile with
     "-D_REENTRANT". Link depends on what version you're using --
     specifically 3.2(x) or 4.0(x). (And for link, watch out for _r
     libraries, e.g., libfoo_r.so or libfoo_r.a if linking static --
     "-threads" or "-pthread" will cause the linker to find and use them
     automatically; but if you roll your own you'll need to look for them
     yourself.)
  2. Digital UNIX 4.0 (and higher) also provides true POSIX threads, if
     you're converting. Compile and link using "cc -pthread" or "cxx
     -pthread". If you can't, compile with "-D_REENTRANT" and link with
     "-lpthread -lexc -lc" (at the end of your list of files). (And, again,
     watch out for _r suffix libraries.)
  3. You mentioned "pthreads_exc.h". Well,  is a header used
     to define the exception-raising variant of the DCE thread (draft 4
     POSIX) API. This conflicts with the  implication of your statement
     "with the posix package", since DCE threads are NOT the same as POSIX
     threads. You cannot use  with POSIX threads.

/---------------------------[ Dave Butenhof ]--------------------------\


================================================================
 Q133: When do you need to know which CPU a thread is on?  

[This is part on an ongoing series of unsolved problems where there
is a lot of "We don't quite know WHY this is happening, but..."  -Bil]

On Sun, 28 Dec 1997, Bil Lewis wrote:

> Jason,
> 
>   That sounds like a very interesting project.  I'm curious about your decision
> to bind threads to CPUs.  You SAY you need to do it, but you don't give any
> proof.  Did you test you system without binding to CPUs?  What kind of results
> did you get when you did?
> 

The threaded version of the system has not been constructed yet however
a non-threaded (ie forked version) has and we have found significant
performance differences between allowing the processes to arbitrarily
migrate between processors and locking the processes to dedicated
processors.

So from that experience it stands to reason that locking threads to
processors would be preferable if we were to attempt to implement
a fully threaded version of the system.

>   I infer from what you say that this is a computationally intensive task.
> Which implies that the threads (or processes) would never migrate to different
> CPUs anyway.  DID they migrate?  I'd very much like to know your experience and
> the performance behavior.

Yes the graphics processes are computationally intensive. It is a standard
technique on multiprocessor SGI's to lock rendering processes to processors.
If they are not locked they will migrate.

The ability to lock threads to processors hasn't been fully implemented
by SGI yet. Currently since threads are bound to their processes, when
the process migrates  the thread gets carried along with it.
I'm guessing that pThreads on the SGI's are being implemented on top of sproc
which is a superset of the capabilities of pthreads. Since sprocs
can be locked to processors I'm hoping soon that the SGI implementation
of pthreads will also inherit that capability.

		==================================
Jason:
> Actually in the work we do (Virtual Reality) we crucially need to know not
> only which processor a thread is running on, but to be able to explicitly
> assign a thread to the processor.

Now I don't see any of that.

You have a set of threads that you want to execute in parallel on an SMP. That's
fine. Lots of people have the same need for all sorts of reasons. That, however,
does NOT mean that you need to know on which processor each thread is running,
much less be able to specify on which processor each thread runs. It just means
you need to be sure that the O/S supports parallel computation.

What you're saying is that you don't trust the O/S scheduling at all, and insist
on controlling it yourself. There are cases where that's valid -- but that's
quite different from staying that your application inherently requires processor
identification or control. It doesn't. In nearly ever case requiring
concurrency/parallelism, you'll be best off trusting the O/S to schedule the
processor resources. And if you find that it's not always trustworthy, tell the
developers, and help them fix it! You, and everyone else, will end up with a
better system.

/---------------------------[ Dave Butenhof ]--------------------------\
==============================================
 Q134: Is any difference between default and static mutex initialization?  

Robert White wrote:

> Venkat Ganti wrote:
>
> > I want to know  whether there is any difference between the following
> > two mutex initializations using pthreads
> >
> > 1.
> >
> > pthread_mutex_t  mp = PTHREAD_MUTEX_INITIALIZER;
> >
> > 2.
> >
> > pthread_mutex_t mp;
> > pthread_mutex_init (&mp, NULL);
> >
> > In this the allocaled memory is zero.
> >
>
> An other way that these two may be different (in addition to the ones
> mentioned by Dave B. in his reply) is that the latter form can have
> different meaning as the program progresses because the default mutex
> behavior of a program can be changed with the set-attribute calls (I
> forget the exact call) when the attribute sepsified in the call is the
> NULL pointer.

You can't change the attribute values of the NULL attributes object. When
you initialize a mutex using NULL, you're asking for default attributes --
those MUST ALWAYS be the same attributes that will be used by a statically
initialized mutex. It doesn't (and can't) matter when the statically
initialized mutex is first used.

> If you use variant 2, you know that the semantics are those in-force at
> the time the statement is executed.  If you use variant 1, it will likely
> have the default semantics in force at the time the mutex is first used.

The only way this could be true is if an implementation provides some
non-portable and non-standard mechanism for modifying the default
attributes. You'd have a hard time convincing me that such an extension
could conform, since pthread_mutex_init specifically requires that the mutex
gain "default" attributes, and the standard requires that the default value
of any attributes (for which the standard doesn't specify a default) must be
specified in the conformance document.

> The manual, if I recall correctly, "strongly suggests" that variant 1
> only be used to initalize staticly allocated mutexes only.  I suspect
> that the above ambiguity is the reason.

Initializing a mutex on the stack is almost always bogus, and will usually
lead to far more trouble than you ever might have imagined. Doesn't matter
whether the mutex is statically initialized or dynamically initialized,
though, except (as always), a static initialization has no choice but to use
the default attributes.

You can't statically initialize a heap mutex, because the language doesn't
allow you to specify an initial value in that case.

/---------------------------[ Dave Butenhof ]--------------------------\

==============================================
 Q135: Is there a timer for Multithreaded Programs?  


From: sullivan@aisg20a.erim.org (Richard Sullivan)
Subject: Re: Timing Multithreaded Programs (Solaris)

bmarker@cco.caltech.edu (Bradley J. Marker) wrote:

>I'm trying to time my multithreaded programs on Solaris with multiple 
>processors.  I want the real world running time as opposed to the total 
>execution time of the programming because I want to measure speedup versus 
>sequential algorithms and home much faster the parallel program is for the user.

Bradly,

  Here is what I wrote to solve this problem (for Solaris anyway).  To
use it just call iobench_start() after any setup that you don't want
to measure.  When you are done measuring call iobench_end().  When you
want to see the statistics call iobench_report().  The output to
stderr will look like this:

Process info:
  elapsed time  249.995
  CPU time      164.446
  user time     152.095
  system time   12.3507
  trap time     0.661235
  wait time     68.6506
  pfs    major/minor    3379/     0
  blocks input/output      0/     0
 
65.8% CPU usage

>>>>>>>>>>>>>>>>>>>>> iobench.h

/*-----------------------------------------------------------------------------
 *
 * Library Name: UTIL
 * Module Name:  iobench
 *
 * Designer:    R. C. Sullivan 
 * Programmer:  R. C. Sullivan
 * Date:        Sep 22, 1995
 *
 * History Of Changes:
 *      Name         Date            Description
 *      ----         ----            -----------
 *      RCS     Jan 17, 1996     Inital release
 *
 * Purpose:
 *   To report resource usage statistics that will be correct for
 * programs using threads on a Solaris system.
 *
 * Notes:
 *
 *-----------------------------------------------------------------------------
 */

extern struct prusage prusagebuf_start, prusagebuf_end;
extern int procfd;
extern double real_time, user_time, system_time, trap_time, wait_time;
extern unsigned long minor_pfs, major_pfs, input_blocks, output_blocks, iochars;

void iobench_start();
void iobench_end();
void iobench_report();

>>>>>>>>>>>>>>>>>>>>> iobench.c

/*-----------------------------------------------------------------------------
 *
 * Library Name: UTIL
 * Module Name:  iobench
 *
 * Designer:    R. C. Sullivan
 * Programmer:  R. C. Sullivan
 * Date:        Sep 22, 1995
 *
 * History Of Changes:
 *      Name         Date            Description
 *      ----         ----            -----------
 *      RCS     Jan 17, 1996     Inital release
 *
 * Purpose:
 *   To report resource usage statistics that will be correct for
 * programs using threads on a Solaris system.
 *
 * Notes:
 *
 *-----------------------------------------------------------------------------
 */

#include 
#include 
#include 
#include 
#include 

#include "iobench.h"

struct stat statbuf;
struct prusage prusagebuf_start, prusagebuf_end;
int procfd;
double real_time, total_real_time, user_time, system_time, trap_time, wait_time;
unsigned long minor_pfs, major_pfs, input_blocks, output_blocks, iochars;

void iobench_start() {
  char pfile[80];

  sprintf(pfile, "/proc/%ld", getpid());
  procfd = open(pfile, O_RDONLY);

  ioctl(procfd, PIOCUSAGE, &prusagebuf_start);
}

void iobench_end() {
  ioctl(procfd, PIOCUSAGE, &prusagebuf_end);
  close(procfd);

  real_time = (double) prusagebuf_start.pr_tstamp.tv_sec +
		(double) prusagebuf_start.pr_tstamp.tv_nsec / NANOSEC;
  real_time = (double) prusagebuf_end.pr_tstamp.tv_sec +
		 (double) prusagebuf_end.pr_tstamp.tv_nsec / NANOSEC - real_time;

  total_real_time = (double) prusagebuf_start.pr_rtime.tv_sec +
		 (double) prusagebuf_start.pr_rtime.tv_nsec / NANOSEC;
  total_real_time = (double) prusagebuf_end.pr_rtime.tv_sec +
		 (double) prusagebuf_end.pr_rtime.tv_nsec / NANOSEC - real_time;

  user_time = (double) prusagebuf_start.pr_utime.tv_sec +
		 (double) prusagebuf_start.pr_utime.tv_nsec / NANOSEC;
  user_time = (double) prusagebuf_end.pr_utime.tv_sec +
		 (double) prusagebuf_end.pr_utime.tv_nsec / NANOSEC - user_time;

  system_time = (double) prusagebuf_start.pr_stime.tv_sec +
		 (double) prusagebuf_start.pr_stime.tv_nsec / NANOSEC;
  system_time = (double) prusagebuf_end.pr_stime.tv_sec +
		 (double) prusagebuf_end.pr_stime.tv_nsec / NANOSEC - system_time;

  trap_time = (double) prusagebuf_start.pr_ttime.tv_sec +
		 (double) prusagebuf_start.pr_ttime.tv_nsec / NANOSEC;
  trap_time = (double) prusagebuf_end.pr_ttime.tv_sec +
		 (double) prusagebuf_end.pr_ttime.tv_nsec / NANOSEC - trap_time;

  wait_time = (double) prusagebuf_start.pr_wtime.tv_sec +
		 (double) prusagebuf_start.pr_wtime.tv_nsec / NANOSEC;
  wait_time = (double) prusagebuf_end.pr_wtime.tv_sec +
		 (double) prusagebuf_end.pr_wtime.tv_nsec / NANOSEC - wait_time;

  minor_pfs = prusagebuf_end.pr_minf - prusagebuf_start.pr_minf;
  major_pfs = prusagebuf_end.pr_majf - prusagebuf_start.pr_majf;
  input_blocks = prusagebuf_end.pr_inblk - prusagebuf_start.pr_inblk;
  output_blocks = prusagebuf_end.pr_oublk - prusagebuf_start.pr_oublk;
/*  iochars = prusagebuf_end.pr_ioch - prusagebuf_start.pr_ioch;*/
}

void iobench_report() {
  fprintf(stderr, "Process info:\n");
  fprintf(stderr, "  elapsed time  %g\n", real_time);
/*  fprintf(stderr, "  total time    %g\n", total_real_time);*/
  fprintf(stderr, "  CPU time      %g\n", user_time + system_time);
  fprintf(stderr, "  user time     %g\n", user_time);
  fprintf(stderr, "  system time   %g\n", system_time);
  fprintf(stderr, "  trap time     %g\n", trap_time);
  fprintf(stderr, "  wait time     %g\n",  wait_time);
  fprintf(stderr, "  pfs    major/minor  %6lu/%6lu\n", major_pfs, minor_pfs);
  fprintf(stderr, "  blocks input/output %6lu/%6lu\n", input_blocks, output_blocks);
/*  fprintf(stderr, "  char inp/out  %lu\n", iochars);*/
  fprintf(stderr, "\n");

/*  fprintf(stderr, "%2.5g Mbytes/sec (real time)\n", iochars /
		 real_time / 1e6);
  fprintf(stderr, "%2.5g Mbytes/sec (CPU time) \n", iochars /
		 (user_time + system_time) / 1e6);*/

  fprintf(stderr, "%2.1f%% CPU usage\n", 100 * (user_time + system_time) /
		 real_time + .05);
}

==============================================
 Q136: Roll-your-own Semaphores   

[For systems that don't support the realtime extensions (where POSIX
semaphores are defined -- they're NOT in Pthreads).]


In article , 
Casper.Dik@Holland.Sun.Com says...
> [[ PLEASE DON'T SEND ME EMAIL COPIES OF POSTINGS ]]
> 
> Bob.Withers@mci.com (Bob Withers) writes:
> 
> >Thanks much for this info.  Unfortunately I need the semaphores for 
> >inter-process mutual exclusion which makes sem_open important.  I'll just 
> >have to stick with SysV semaphores until we can move to 2.6.
> 
> 
> Well, you can mmap a semaphore in a file if you wish.
> 

Well you sure can and, believe it or not, I actually thought of it before 
I read your post.  My code has not been thoroughly tested but I'm posting 
it here in the hopes that it will be of help to someone else.  Either 
that or I'm just a glutten for criticism.  :-)

Casper, thanks much for your help.

Bob


#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 


sem_t *sem_open(const char *name, int oflag, ...)
{
    auto     int                need_init = 0;
    auto     int                val = 1;
    auto     int                fd;
    auto     sem_t *            sem = (sem_t *) -1;
    auto     struct stat        st;

    /* -----------------2/11/98 2:12PM-------------------
     * open the memory mapped file backing the shared
     * semaphore to see if it exists.
     * --------------------------------------------------*/
    fd = open(name, O_RDWR);
    if (fd >= 0)
    {
        /* -----------------2/11/98 2:13PM-------------------
         * the semaphore already exists, it the caller
         * specified O_CREAT and O_EXCL we need to return
         * an error to advise them of this fact.
         * --------------------------------------------------*/
        if ((oflag & O_CREAT) && (oflag & O_EXCL))
        {
            close(fd);
            errno = EEXIST;
            return(sem);
        }
    }
    else
    {
        auto     int                sem_mode;
        auto     va_list            ap;

        /* -----------------2/11/98 2:14PM-------------------
         * if we get here the semaphore doesn't exist.  if
         * the caller did not request that ir be created then
         * we need to return an error.  note that errno has
         * already been set appropriately by open().
         * --------------------------------------------------*/
        if (0 == (oflag & O_CREAT))
            return(sem);

        /* -----------------2/11/98 2:15PM-------------------
         * ok, we're going to create a new semaphore.  the
         * caller should've passed mode and initial value
         * arguments so we need to acquite that data.
         * --------------------------------------------------*/
        va_start(ap, oflag);
        sem_mode = va_arg(ap, int);
        val = va_arg(ap, int);
        va_end(ap);

        /* -----------------2/11/98 2:16PM-------------------
         * create the semaphore memory mapped file.  if this
         * call returns an EEXIST error it means that another
         * process/thread snuck in and created the semaphore
         * since we discovered it doesn't exist above.  we
         * don't handle this condition but rather return an
         * error.
         * --------------------------------------------------*/
        fd = open(name, O_RDWR | O_CREAT | O_EXCL, sem_mode);
        if (fd < 0)
            return(sem);

        /* -----------------2/11/98 2:18PM-------------------
         * set flag to remember that we need to init the
         * semaphore and set the memory mapped file size.
         * --------------------------------------------------*/
        need_init = 1;
        if (ftruncate(fd, sizeof(sem_t)))
        {
            close(fd);
            return(sem);
        }
    }

    /* -----------------2/11/98 2:19PM-------------------
     * map the semaphore file into shared memory.
     * --------------------------------------------------*/
    sem = (sem_t *) mmap(0, sizeof(sem_t), PROT_READ | PROT_WRITE,
                            MAP_SHARED, fd, 0);
    close(fd);
    if (sem)
    {
        /* -----------------2/11/98 2:19PM-------------------
         * if the mapping worked and we need to init the
         * semaphore, do it now.
         * --------------------------------------------------*/
        if (need_init && sem_init(sem, 1, val))
        {
            munmap((caddr_t) sem, sizeof(sem_t));
            sem = 0;
        }
    }
    else
    {
        sem = (sem_t *) -1;
    }

    return(sem);
}


int sem_close(sem_t *sem)
{
    return(munmap((caddr_t) sem, sizeof(sem_t)));
}


int sem_unlink(const char *name)
{
    return(remove(name));
}

==============================================
 Q137: Solaris sockets don't like POSIX_C_SOURCE!  

A little known requirement in Solaris is that when you define POSIX_C_SOURCE,
you must also define __EXTENSIONS__ when including sys/socket.h.  Hence,
your file should look like this:


#define _POSIX_C_SOURCE 199506L 
#define __EXTENSIONS__

#include 
#include 
#include 
...
		================

That's because POSIX_C_SOURCE of 1995 vintage doesn't include
socket calls.

The feature macros are *exclusion* macros, not *inclusion* macros.

By default, you will get everything.

When you define something, you get *only* that something.
(Unless you also define __EXTENSIONS__)

This is slightly different in cases where the behaviour is
modified by the macro as in some socket calls.

Casper
		======================
From: robinson@jetsun.eng.sun.com (David Robinson)

The gratuitous use of non-POSIX conforming typedefs in headers is the
root cause. (Should use ushort_t not u_short and uint_t not u_int)
When defining POSIX_C_SOURCE it says use only strictly POSIX conforming
features, typedefs thus can only end in _t.

Good news is that 90+% of the offending headers are fixed in 2.7.

	-David
		================
% A question... should I use -mt or -D_POSIX_C_SOURCE=199506L to compile
% a pthread program on Solaris 2.6?  If I use the latter even the most simple
% socket program won't compile.  For example,

Well, these do different things. -mt sets up the necessary macro definitions
for multi-threading, and links with the appropriate libraries. _POSIX_C_SOURCE
tells the compiler that your application is supposed to strictly conform
to the POSIX standard, and that the use of any non-POSIX functions or types
that might be available on the system should not be allowed.

The advantage of this is that, when you move to another system which provides
POSIX support, you are assured of your program compiling, however this
requires some work up-front on your part.

So the answer to your question is that you should use -mt to tell the
compiler your application is multi-threaded, and use _POSIX_C_SOURCE only
if your application is intended to conform strictly to POSIX.

HP's compiler is quite frustrating in this regard, since it assumes by
default that your application is k&r C. If you use the -Aa option to
tell it your application is ANSI C, it doesn't allow any functions which
aren't defined by ANSI. I always end up using -Ae to tell the compiler to
get stuffed and just compile my program with whatever's on the system, and
I port to HP last after a big change.

==============================================
 Q138: The Thread ID changes for my thread!  

	I'm using IRIX6.4 threads and MPI-SGI but I'm having
strange problems. To analyse and even debug my program I begun
to write some "similar behaviors" programs very simples, and I
detected a strange thing. Anybody can says me if I doing a
mistake or if was a problem with IRIX 6.4 systems.

	The problems is: WHEN I CHANGE THE THREAD PRIORITY
THE THREAD ID IS ALSO CHANGED. As you can imagine I have a
lot of problems when I try joining them.

	- If I use only threads call, I commented MPI calls,
the program works fine even if I link with mpi library.
The program changes the main thread priority, and after,
it creates 10 threads with other priorities. Threads Id
are sequential.

	- If I use threads and MPI call (only MPI_Init,
Comm_size, Comm_rank and Finalize) the SAME program does
a change on main thread id after prioriry changing.

	- Another thing: in the first case, on my execution,
thread id begun with id=10000 and the other are sequential
after 10000. In the second case, thread id begun with 10000
and after priority change it assumes id=30000.

	ANYBODY can explain me? TIA.

THE CODE IS:


#include        
#include        
#include        
#include        

pthread_attr_t attr;
pthread_mutex_t mutex;
pthread_cond_t cond;
int xcond=0;
int size,rank;

void *Slaves(void *arg)
{
    int i,j;

    pthread_mutex_lock(&mutex);
    while(xcond==0)
       pthread_cond_wait(&cond,&mutex);
                         
    pthread_mutex_unlock(&mutex);
    printf("Size: %d Rank: %d    Thread Id %x\n", size, rank,
pthread_self());
    fflush(stdout);
}

int main (int argc, char **argv)
{
    int i,k=10,r;
    pthread_t *ThreadId;  
    struct sched_param params;
    int sched;

    printf("THREAD MAIN BEFORE MPI INIT %x\n", pthread_self());
/*   
 *  This lines are commented to see if MPI calls influence over
 *  system behavior.
 
    MPI_Init(&argc, &argv);
    MPI_Comm_size(MPI_COMM_WORLD, &size);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);

    printf("THREAD MAIN AFTER MPI INIT %x\n", pthread_self());

 */

/*
 * If I called MPI the main thread id will be changed after
 * this lines. When I just left MPI initialisation main thread
 * has the same ID that it had before, the problem arrives
 * from this point
 */

    params.sched_priority=20;
    pthread_setschedparam(pthread_self(), SCHED_RR, ¶ms);

    printf("THREAD MAIN AFTER  PRIO CHG %x\n", pthread_self());


    
    if (argc==2)
       k=atoi(argv[1]);

    ThreadId= (pthread_t *) malloc(k*sizeof(pthread_t));

    pthread_attr_init(&attr);
    pthread_mutex_init(&mutex, NULL);
    pthread_cond_init(&cond, NULL);

    printf("\n Creating %d threads - Main thread is %x  \n", k,
pthread_self());

    for(i=0; i != k; i++) {
        pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED); 
        params.sched_priority=21;
        pthread_attr_setschedparam(&attr, ¶ms);
        r=pthread_create(&ThreadId[i], &attr, Slaves, NULL);
        if (r!=0) {
           printf("Error on thread creation! \n");
           exit(0);
        }
    }

    xcond=1;
    pthread_cond_broadcast(&cond);

/*
 * It was to force threads execution, but this is not necessary
  
    for(;;) sched_yield(); 
 */

    for(i=0; i != k; i++) {
        r=pthread_join(ThreadId[i], NULL);
        if (r!=0) {
           printf("Error on joining threads...\n");
           exit(0);
         }
    }
    printf(" Thead MAIN with id %x terminating...\n", pthread_self());
/*
    MPI_Finalize();
*/
}


==============================================
 Q139:  Does X11 support multithreading ?  
> > I am developing a multithreaded app under Solaris 2.5.1 (UltraSPARC),
> > using mixed Motif and X11, and i wonder if  someone can help me
> > answering some question:
> >
> > Does X11 support multithreading ?
> 
>   Well, kinda.  But...

Kinda?

The X Consortium releases of R6.x can be build MT-safe.

You can tell if you have R6 when you compile by checking the
XlibSpecificationRelease or XtSpecificationRelease feature test macros.
If they are >5 then your implementation may support threaded
programming. Call XInitThreads and/or XtToolkitInitializeThreads to find
out if your system's Xlib and Toolkit Intrinsics (libXt) really do
support threaded programming.

> 
> > Does Motif do the same ?
> 
>   No.  It's not thread-safe.

Motif 2.1 IS MT-safe.

> 
> > Can different threads open their own window and listen to their own
> > XEvents ? How could they do that ? [XNextEvent() can't specify a window
> > handle !].
> 
>   You don't.  Your main loop listens for any event, and then decides what
> to do with it.  Perhaps it hands off a task to another thread.

You can. Each thread could open a separate Display connection and do
precisely what Daniele asks. Even without separate Display connections,
the first thread to call XNextEvent will lock the Display, and the
second thread's call to XNextEvent will block until the first thread
releases its lock. But you can't guarantee which thread will get a
particular event, so in the trivial case you can't be assured that one
thread will process events solely for one window.

> 
>   Take a look at the FAQ for the threads newsgroup (on the page below).  That
> will help a bit.  You may also want to get "Multithreaded Programming with Pthreads"
> which has a section on exactly this, along with some example code.  (No one else
> talks about this, but I thought it important.)

I recommend reading the Xlib and Xt specifications, which are contained
in each and every X Consortium release -- available at
ftp://ftp.x.org/pub/, or you can get ready-to-print PostScript of just
the Xlib and Xt specs from ftp://ftp.x.org/pub/R6.3/xc/doc/hardcopy.
==============================================
 Q140: Solaris 2 bizzare behavior with usleep() and poll()  
>Jeff Denham wrote:
>> 
>> Adam Twiss wrote:
>> 
>> > You really really don't want to use usleep() in a threaded environment.
>> > On some platforms it is thread safe, but on Solaris it isn't.  The
>> > affects are best described as "unpredictable", but I've seen a usleep()
>> > call segv because it was in a theaded program on Solaris.
>> >
>> > You want to use nanosleep() instaed.
>> >
>> > Adam
>> 
>> I've found that poll(0, 0, msec-timeout-value)
>> works pretty well. Is there significant overhead calling
>> poll in this manner?
>
>It's not uncommon to use poll() or select() for sleeping. Works
>great.

I've seen an occasional bug in Solaris 2.6 where poll() will fail to
restore a pre-existing SIGALRM handler when it returns.  The sequence
is:

	sigaction(SIGALRM,...);
	alarm();
	...
	poll(0, 0, timeout);
	...
	(program exits with "Alarm clock" error)

Looking at the truss output, poll() appears to be the only place
after the initial sigaction where the handler for SIGALRM is changed.
The failure is difficult to reliably reproduce, but I've seen it
happen about 10% of the time.  It only happens on 2.6; this same code
works fine on Solaris 2.5.1, AIX 3.2.5 and 4.2.1, HP-UX 9.04 and
10.10, and SCO OpenServer 5.0.4.

The same thing happens with usleep.  I haven't tried nanosleep.

The program in question is single-threaded.

I haven't had the chance to pursue this problem yet; there may be a
fix for it, or it may be some really subtle application bug.

Michael Wojcik						mww@microfocus.com
AAI Development, Micro Focus Inc.
Department of English, Miami University

Q: What is the derivation and meaning of the name Erwin?
A: It is English from the Anglo-Saxon and means Tariff Act of 1909.
-- Columbus (Ohio) Citizen
==============================================
 Q141: Why is POSIX.1c different w.r.t. errno usage?  

Bryan O'Sullivan wrote:

> d> It's an issue because that implementation is "klunky" and, more
> d> precisely, inefficient.
>
> I must admit that optimising for uncommon error cases does not make
> much sense to me.

Sure. In my sentence, I would have to say that "klunky" was a more
important consideration than "inefficient".

However, use of errno is NOT strictly in "uncommon error cases". For
example, pthread_mutex_trylock returns EBUSY when the mutex is locked.
That's a normal informational status, not an "uncommon error". Similarly,
pthread_cond_timedwait returns ETIMEDOUT as a normal informational
status, not really an "error". There are plenty of "traditional" UNIX
functions that are similar. It's certainly not universal, but "uncommon"
is an overstatement.

> d> Still, why propagate the arcane traditions, just because they
> d> exist?
>
> Because they are traditions.  I think there is some non-trivial value
> in preserving interface consistency - principle of least surprise, and
> all that - and 1003.1c violates this for no particularly good reason.

Let's just say that the working group (a widely diverse and contentious
bunch) and the balloting group (an even larger and more diverse group)
were convinced that the reasons were "good enough". Arguing about it at
this point serves no purpose.

> d> Overloading return values with "-1 means look somewhere else for an
> d> error" is silly.
>
> Sure, it's silly, but it's the standard way for library calls on Unix
> systems to barf, and what 1003.1c does is force programmers to plant
> yet another gratuitous red flag in their brains, with wording similar
> to "hey!  everything else works in such-and-such a way, but *this* is
> *different*!".  I have enough red flags planted in my brain at this
> point that it resembles a pincushion, and I would gladly sacrifice a
> few to ugliness if that ugliness were at least consistent.

UNIX isn't even very consistent about that. Some return -1. Some have
symbolic definitions that "are usually" -1 but needn't be (at least in
terms of guaranteed portability and conformance). Some return NULL. Some
set errno and some don't, requiring that YOU set errno before making the
call if you care WHY it failed (e.g., sysconf).

Hey, even if there was a "C" in "UNIX", it would most definitely NOT
stand for "consistency". Adding threads to UNIX disturbed a lot of
cherished traditions... far more than most people are willing to
acknowledge until they stumble over the shards of the old landscape.
There was good reason for this, though, and the net result is of
substantial benefit to everyone. While the changes to errno may be one of
the first differences people notice, "in the scheme of things", it's
trivial. If it even raises your awareness that "something's different",
maybe it'll save a few people from some bad mistakes, and to that extent
it's valuable even merely as a psychological tool.

Hey... count your mixed blessings, Bryan. I would have reported errors by
raising exceptions, if there'd been any hope at all of getting that into
POSIX. ;-)

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/

==============================================
 Q142: printf() anywhere AFTER pthread_create() crashes on HPUX 10.x  

>I've spend the last couple of days trying to track down an
>annoying problem when running a multithreaded program on
>HPUX build against DCE Threads...
>
>If I call printf anywhere AFTER pthread_create had been executed to start
>a thread, when my application ends I get thrown out of my rlogin shell.

We experienced a similar problem about a year ago.  It was a csh bug,
and a patch from HP fixed it.

==============================================
 Q143: Pthreads and Linux  

Wolfram Gloger wrote:


> You should always put `-lpthread' _after_ your source files:
>
> % gcc the_file.cpp -lpthread
>
> Antoni Gonzalez Ciria  writes:
>
> > When I compile this code with gcc ( gcc -lpthread the_file.cpp) the 
> > program executes fine, but doing so with g++( g++ -lpthread the_file.cpp)
> > the progran crashes, giving a Segmentation fault error.
> >
> > #include 
> >
> > void main(){
> >     FILE * the_file;
> >     char sBuffer[32];
> >
> >     the_file=fopen("/tmp/dummy","rb");
> >     fread( sBuffer, 12, 1, the_file);
> >     fclose( the_file);
> >
> > }
>
> Using `g++' as the compiler driver always includes the `libg++'
> library implicitly in the link.  libg++ has a lot of problems, and is
> no longer maintained (in particular, it has a global constructor
> interfering with glibc2, if I remember correctly).  If you really need
> it, you must get a specially adapted version for Linux/egcs/gcc-2.8.
>
> If you don't need libg++, please use `c++' or `gcc' as your compiler
> driver, and use the libstc++ library shipped with egcs or seperately
> with gcc-2.8 (`c++' will link libstdc++ in implicitly).
>
> When I just tested your program with egcs-1.0.1 and glibc-2.0.6, it
> crashed at first (it fails to check the fopen() result for being
> NULL), but after creating a /tmp/dummy file it ran perfectly, even
> after compiling it with `c++ the_file.cpp -lpthread'.
>
> Regards,
> Wolfram.

  The -pthread option takes care of everything:
     adds  -D_REENTRANT  during the cpp pass, and
     adds  -lpthread during the link-edit.
  This option has been around for a while. I'm not sure
   it's working for all ports. At least for the x86 AND glibc.
   You may want to take a look at the spec file (gcc -v)

jms.
==============================================
 Q144: DEC release/patch numbering  

It was after 4.0B, and 4.0C is just 4.0B with new hardware support. (If you
install a "4.0C" kit on any hardware that doesn't need the new support, it
will even announce itself as 4.0B.) Although this is not true of all
components, DECthreads policy has been to keep 4.0 through 4.0C identical --
we have always submitted any patches to the 4.0 patch stream, and propagated
the changes through the other patch streams, releasing "functionally
identical" patches that are simply built in the appropriate stream's
environment. (But note that all future patches for the 4.0 - 4.0C stream
will be available only on 4.0A and later... 4.0 is no longer supported.)

The changes are in source in 4.0D and later.

/---------------------------[ Dave Butenhof ]--------------------------\
==============================================
 Q145: Pthreads (almost) on AS/400  
Fred A. Kulack wrote:

> Hi All.
> You may or may not know, that new in the v4r2m0 release of OS/400 is support
> for kernel threads. Most notably the support for threads is available via
> native Java, but we've also implemented a Pthreads library that is 'based on'
> Posix and Unix98
> Pthreads.

Thank you very much for this update. I'm not likely to ever have occasion or
need to use this, but I like to keep track of who's implemented threads (and
how, and which threads). If nothing else, it provides me with more information
to answer the many questions I get on threading.

> The implementation claims no compliance because there are some differences
> and we haven't implemented all of the APIs. We do however duplicate the
> specification for the APIs that are provided, and we have quite a full set
> of APIs.

Yeah, I understand that one. Same deal on OpenVMS. Most of the APIs are there,
and do more or less what you'd expect -- but VMS isn't POSIX, and some parts
just don't fit. Congratulations on "doing your best". (At least, since you say
"we", I'm making the liberal assumption that you bear some personal
responsibility for this. ;-) )

> Anyone whose interested, can take a look at
> http://www.softmall.ibm.com/as400/threads

==============================================
 Q146:   

==============================================
 Q147: Thread create timings  

Matthew Houseman  writes:

Thought I'd throw this into the pyre. :)  I ran the thread/process create
stuff on a 166MHz Pentium (no pro, no mmx) under NT4 and Solaris x86 2.6:


NT spawn                240s    24.0  ms/spawn
Solaris spawn (fork)    123s    12.3  ms/spawn  (incl. exec)
Solaris spawn (vfork)    95s     9.5  ms/spawn  (incl. exec)

Solaris fork             47s     4.7  ms/fork
Solaris vfork                    0.37 ms/vfork  (37s/100000)

NT thread create         12s     1.2  ms/create
Solaris thread create            0.11 ms/create (11s/100000)


As you can see, I tried both fork() and vfork(). When doing an immediate
exec(), you'd normally use vfork(); when just forking, fork() is usually
what you want to use (or have to use).

Note that I had to turn the number of creates up to 100000 for vfork
and thread create to get better precision in the timings.


To remind you, here are greg's figures (on a Pentium MMX 200MHz):

>NT Spawner (spawnl):            120 Seconds (12.0 millisecond/spawn)
>Linux Spawner (fork+exec):       57 Seconds ( 6.0 millisecond/spawn)
>
>Linux Process Create (fork):     10 Seconds ( 1.0 millisecond/proc)
>
>NT Thread Create                  9 Seconds ( 0.9 millisecond/thread)
>Linux Thread Create               3 Seconds ( 0.3 millisecond/thread)


Just for fun, I tried the same thing on a 2 CPU 170MHz Ultrasparc.
I leave it to someone else to figure out how much of this is due to
the two CPUs... :)

Solaris spawn (fork)            84s     8.4  ms/spawn  (incl. exec)
Solaris spawn (vfork)           69s     6.9  ms/spawn  (incl. exec)

Solaris fork                    21s     2.1  ms/fork
Solaris vfork                           0.17 ms/vfork  (17s/100000)

Solaris thread create                   0.06 ms/create (6s/100000)


==============================================
 Q148: Timing Multithreaded Programs (Solaris)  

From: sullivan@aisg20a.erim.org (Richard Sullivan)

>I'm trying to time my multithreaded programs on Solaris with multiple 
>processors.  I want the real world running time as opposed to the total 
>execution time of the programming because I want to measure speedup versus 
>sequential algorithms and home much faster the parallel program is for the user.

Bradly,

  Here is what I wrote to solve this problem (for Solaris anyway).  To
use it just call iobench_start() after any setup that you don't want
to measure.  When you are done measuring call iobench_end().  When you
want to see the statistics call iobench_report().  The output to
stderr will look like this:

Process info:
  elapsed time  249.995
  CPU time      164.446
  user time     152.095
  system time   12.3507
  trap time     0.661235
  wait time     68.6506
  pfs    major/minor    3379/     0
  blocks input/output      0/     0
 
65.8% CPU usage

The iobench code is included in the program sources on: index.html.
==============================================
 Q149: A program which monitors CPU usage?  

> >Ok, I've tried some web searches and haven't found anything I like the
> >look of.  What I'm after is a program which runs in the background and
> >monitors (primarily) CPU usage for our web server (an Ultra-1 running
> >Solaris 2.6).  However, all the programs I've found are about 2 years
> >old and/or don't run on 2.6.
> >
> >I've seen top, but it doesn't really do what I want; I'd like to have
> >the output from the program as a %cpu usage for each hour (or some
> >other arbitrary time period) stored as a log file or, ideally, as a
> >graph (in some graphics format, eg, .gif).
> 
> Sounds like what sar does, and it comes with 2.6 - to enable recording
> data for it, just uncomment the lines in /etc/init.d/perf and the
> crontab for the 'sys' account.

From what I've read on the product, sounds like 'spong' might be what
you need. I've downloaded it, but haven't had time to install and set up
yet. Try:

http://strobe.weeg.uiowa.edu/~edhill/public/spong/
==============================================
 Q150: standard library functions: whats safe and whats not?  

From: rstevens@noao.edu (W. Richard Stevens)
Subject: Re: standard library functions: whats safe and whats not?
Date: 17 Feb 1998 14:19:28 GMT

> 1.  Which of the standard C library functions are thread-safe and
> which aren't?  For example, I know that strtok() is un-safe, I can
> infer that from its functionality, but what about the thousands of
> other library calls? I don't want to examine each one individually
> and make guesses about thread safety.
>
> Is there a list somewhere of what's safe and whats not?

Page 32 of the 1996 Posix.1 standard says "All functions defined by
Posix.1 and the C standard shall be thread-safe, except that the following
functions need not be thread-safe:

	asctime()
	ctime()
	getc_unlocked()*
	getchar_unlocked()*
	getgrid()
	getgrnam()
	getlogin()
	getpwnam()
	getpwuid()
	gmtime()
	localtime()
	putc_unlocked()*
	putchar_unlocked()*
	rand()
	readdir()
	strtok()
	ttyname()"

Note that a thread-safe XXX_r() version of the above are available,
other than those with an asterisk.  Also note that ctermid() and
tmpnam() are only thread-safe if a nonnull pointer is used as an
argument.

	Rich Stevens


		================

POSIX and ANSI C specify only a small part of the "traditional UNIX
programming environment", though it's a start. The real danger in reading the
POSIX list quoted by Rich is that most people don't really know what's
included. While an inclusive list would be better than an exclusive list,
that'd be awfully long and awkward.

The Open Group (OSF and X/Open) has extended the Single UNIX Specification
(also known as "SPEC1170" for it's 1,170 UNIX interfaces, or UNIX95) to
include POSIX.1b-1993 realtime, POSIX.1c-1995 threads, and various
extensions. It's called the Single UNIX Specification, Version 2; or UNIX98.
Within this calendar year, it's safe to assume that most UNIX versions
currently branded by The Open Group (as XPG3, UNIX93, UNIX95) will extend
their brand validation to UNIX98.

The interfaces specification part of the Single UNIX Specification, Version 2
(known as XSH5), in section 2.8.2, "Thread-safety", specifies that all
interfaces defined by THIS specification will be thread-safe, except for "the
following". There are two explicit lists, and one implicit. One is the POSIX
list already quoted by Rich Stevens. The second is an additional list of
X/Open interfaces:

basename      dbm_open    fcvt        getutxline    pututxline
catgets       dbm_store   gamma       getw          setgrent
dbm_clearerr  dirname     gcvt        l64a          setkey
dbm_close     drand48     getdate     lgamma        setpwent
dbm_delete    ecvt        getenv      lrand48       setutxent
dbm_error     encrypt     getgrent    mrand48       strerror
dbm_fetch     endgrent    getpwent    nl_langinfo
dbm_firstkey  endpwent    getutxent   ptsname
dbm_nextkey   endutxent   getutxid    putenv

The implicit list is a statement that all interfaces in the "Legacy" feature
group need not be thread-safe. From another section, that list is:

advance       gamma          putw        sbrk          wait3
brk           getdtablesize  re_comp     sigstack
chroot        getpagesize    re_exec     step
compile       getpass        regcmp      ttyslot
cuserid       getw           regex       valloc
        
loc1          __loc1         loc2        locs

Obviously, this is still an exclusive list rather than inclusive. But then,
if UNIX95 had 1,170 interfaces, and UNIX98 is bigger, an inclusive list would
be rather awkward. (And don't expect ME to type it into the newsgroup!)

On the other hand... beware that if you've got a system that doesn't claim
conformance to POSIX 1003.1c-1995 (or POSIX 1003.1-1996, which includes it),
then you're not guaranteed to be able to rely even on the POSIX list, much
less the X/Open list. It's reasonable to assume that any implementation's
libpthread (or equivalent, though that name has become pretty much defacto
standard) is thread-safe. And it's probably reasonable to assume, unless
specified otherwise, that "the most common" bits of libc are thread-safe. But
without a formal statement of POSIX conformance, you're just dealing with
"good will". And, even at that, POSIX conformance isn't validated -- so
without validation by the UNIX98 branding test suite, you've got no real
guarantee of anything.

/---------------------------[ Dave Butenhof ]--------------------------\
==============================================
 Q151: Where are semaphores in POSIX threads?  

David McCann wrote:

> Jan Pechanec wrote:
> >
> > Hello,
> >
> >         I have a summary of POSIX papers on threads, but there in no
> > imformation about semaphores (just conditional vars, mutexes). *NO*
> > pthread_semaphoreinit() etc.
> >
> >         In some materials, there is information on sem_wait(), sem_send() (or
> > sm. like that), but is it for threads (or just processes)?
>
> I think this whole discussion has digressed from Jan's original question
> above. Yes, there are sem_* calls in Solaris 2.5 (and 2.4 IIRC); you
> just need to link with -lposix4 or whatever to get them. But these are the
> *POSIX.1b* semaphores, which are *process-based* semaphores. They have
> nothing to do with threads.
>
> Now what Jan wants here is semaphore calls for *POSIX.1c*, i.e. POSIX
> threads. Now, IIRC, the sem_* calls are NOT specified in POSIX.1c, but
> rather their behaviour in MT programs has been clarified/refined in XPG5
> (Unix98) which allows you to use semaphores to synchronize threads and/or
> processes, depending on how you use them.

Not quite true. Yes, XSH5 (the "system interfaces" part of XPG5) says this; but it
does so because POSIX 1003.1-1996 says so, not because it's added something to
POSIX.

In fact, POSIX 1003.1b semaphores were designed by the same working group that did
1003.1c, and while 1003.1b-1993 was approved and published first (and therefore
couldn't mention threads), several aspects of 1003.1b were designed to work with
threads. For example, there are "realtime" extended versions of the 1003.1c
sigwait functions (sigtimedwait and sigwaitinfo). (The interfaces are slightly
incompatible because they return -1 and set errno on errors, rather than returning
an error code: that's because 1003.1c removed the use of errno AFTER 1003.1b was
finalized.)

Additionally, the sem_init function was designed with a parameter corresponding to
the "pshared" attribute of mutexes and condition variables. For 1003.1b, the only
supported value was 1, meaning "process shared". 1003.1c amended the sem_init
description to specify in addition that the value 0 meant "process private", for
use only between threads within the process. (But also note that it's perfectly
reasonable to create a "process shared" semaphore and then use it only between
threads within the process -- it may be less efficient on some implementations,
but it does the same thing.)

> Solaris 2.5 is not Unix98-conformant; the confusion arises because it
> *does* appear to be compliant with POSIX.1b and POSIX.1c (somebody at Sun can
> surely verify this). From what's been said here, I assume 2.6 is either Unix98-
> compliant, or at least contains the MT extensions to POSIX of Unix98.

Solaris 2.5 supports (most of) 1003.1b and 1003.1c, although there were a few
omissions and a few interpretation errors. (Like any other implementation.) This,
however, is not one of them. Solaris 2.5 does NOT define _POSIX_SEMAPHORES in
, which is the way an implementation should advertise support for POSIX
semaphores. Therefore, while it may not implement all capabilities described by
1003.1b and 1003.1c, it doesn't (in this case, anyway) violate the standard. If
you're using POSIX semaphores (even if they seem to work) on Solaris 2.5, then
your application is not "strictly conforming", and if you're subject to any
incompatibilities or porting problems, that's your fault, not the fault of
Solaris. IT says they're not there.

(And, yes, POSIX specifically allows an implementation to provide interfaces while
claiming they're not there; and if it does so, it's not obligated to provide
strict conformance to the standard's description of those interfaces. This is what
Solaris 2.5 should have done, also, with the _POSIX_THREAD_PRIORITY_SCHEDULING
option, since it doesn't completely implement those interfaces.)

Presumably, Solaris 2.6 (though I don't have a system handy to check) DOES define
_POSIX_SEMAPHORES.

> At any rate, you can't use the sem_* calls for thread synchronization in
> 2.5; you get ENOSYS in MT programs. I know, I've tried it (on 2.5.1).
> AFAIK, single-threaded programs linked with -lposix4 work fine, but as I
> said above, they're only for process-based semaphores. So if you want to use the
>
> sem_* calls for thread-synchronization on Solaris, you have to go to 2.6.

First off, other replies have indicated that it's actually libthread, not
libposix4, that provides "working" (though not complete) POSIX semaphores. Most
likely, these semaphores would work with the "pshared" parameter set to either 0
(process) or 1 (cross-process). However, in any case, if you've got something that
can synchronize between processes, you should expect that it can synchronize
between threads as well, though there may be alternatives that are more efficient
on some implementations. (E.g., a pshared mutex will usually be more expensive to
lock or unlock than a private mutex.) (Such a difference in efficiency is less
likely for semaphores, since POSIX already requires that sem_post be async-signal
safe, which means it's far better and easier to keep the implementation inside the
kernel regardless of the pshared argument.)

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/

==============================================
 Q152: Thread & sproc (on IRIX)  

In article <34EAF75A.4BF2@loria.fr>,
Yann Boniface   wrote:
>I'm having trouble while using threads and processus on a massive
>parallel machine (SGI).
>The processus creation is OK (sproc (myFunction, PR_SADDR, arg)) as long
>as I don't use pthread library. If I compile the program with the flag
>-lpthread, processus creation didn't work any more, even if I don't
>explicity use thread functions (errno is then ENOTSUP)

You shouldn't mix pthreads and sprocs. You should stick with one or
the other (IMHO pthreads are preferable).
-- 
Planet Bog -- pools of toxic chemicals bubble under a choking
atomsphere of poisonous gases... but aside from that, it's not
much like Earth.
==============================================
 Q153:  C++ Exceptions in Multi-threaded Solaris Process  

Jeff Gomsi  writes:

> We are running a large multi-threaded C++ (C++ 4.2 patch 
> 104631-03) application under Solaris (SunOS 5.5.1 
> Generic_103640-14) on a 14 processor Ultra-Enterprise and 
> observe the following problem.
> 
> The application runs fine single-threaded, but when run
> multi-threaded, throwing a C++ exception can (evidently) 
> cause memory corruption which leads to a SIGSEGV core
> dump. A diagnostic version of the new operator seems to
> reveal that C++ is unwinding things improperly and possibly
> calling destructors which should not be called.
> 
> Does anyone have any ideas on this?

The last time I looked at the patch list for the C++ 4.2, I noticed a
mention of a bug stating that exceptions were not thread safe.  There was
no further description of this bug that I could find.  However, it
supposedly is addressed by one of the later patches. Try upgrading your
patch to -04 or -05....

- Chris

Make sure you have the libC patch 101242-13.
==============================================
 Q154:  SCHED_FIFO threads without root privileges  ?  

Laurent Deniel wrote:

> Hi,
>
>  Is there a way to create threads that have the SCHED_FIFO scheduling
>  without root privileges (in system contention scope) ? by for instance
>  changing a kernel parameter (Digital UNIX 4.0B & D or AIX 4.2) ?
>
>  Thanks in advance,

In Digital UNIX 4.0, using process contention scope (PCS) threads,
any thread can set FIFO policy; further, it can set any priority.
Why? Because the policies and priorities for PCS threads affect only
the threads in the containing process. PCS FIFO/63 threads are really
important in relation to other PCS threads in the process, but have no
influence on the scheduling of other threads in other processes.
The aspect is controlled by the policies and priorities of  the kernel
scheduling entities (VPs -- virtual processors) underlying the PCS
threads, and those characteristics are unaffected by the POSIX
scheduling interfaces.

On V4.0D, newly released, system contention scope (SCS) threads
are supported. Their policies and priorities are by definition seen
by the kernel scheduler and are therefore subject to privilege
restrictions. In short, you can set SCS threads to FIFO or RR policy
without  privilege on V4.0D, but FIFO threads cannot exceed POSIX
prio 18 and RR threads cannot exceed 19. Regardless of this
"limitation," it gives you plenty or rope to hang yourself with!

__________________________________________________
Jeff Denham (jdenham@brighttiger.com)
==============================================
 Q155: "lock-free synchronization"  

> I recently came across a reference to "lock-free synchronization" (in
> Taligent's Guide to Designing Program's.)  This document referred to
> research that was looking at using primitive atomic operations to build more
> complex structures in ways that did not require locking.
>
> I'm interested in exploring this topic further and would be greatful if
> anyone could supply references.
>
> Regards,
> Daniel Parker
>
>

Check out the following references --

  M. Herlihy, "Wait free Synchronization," ACM Transactions on Programming
Languages and Systems, Vol 13, No 1, 1991, pp. 124-149.

  M. Herlihy, "A Methodology for Implementing Highly Concurrent Data
Objects," same journal as above, Vol 15, No. 5, 1993, pp. 745 --770.

They should provide a starting point.

==============================================
 Q156: Changing single bytes without a mutex  

Tim Beckmann wrote:

> David Holmes wrote:
> >
> > I thought about this after posting. An architecture such as Bil describes
> > which requires non-atomic read/mask/write sequences to update variables of
> > a smaller size than the natural word size, would be a multi-threading
> > nightmare. As you note above two adjacent byte values would need a common
> > mutex to protect access to them and this applies even if they were each
> > used by only a single thread! On such a system I'd only want to program
> > with a thread-aware language/compiler/run-time.
> >
> > David
>
> David,
>
> My thoughts exactly!
>
> Does anyone know of a mainstream architecture that does this sort of
> thing?

Oh, absolutely. SPARC, MIPS, and Alpha, for starters. I'll bet most other RISC
systems do it, too, because it substantially simplifies the memory subsystem
logic. And, after all, the whole point of RISC is that simplicity means speed.

If you stick to int or long, you'll probably be safe. If you use anything
smaller, be sure they're not allocated next to each other unless they're under
the same lock.

I wrote a long post on most of the issues brought up in this thread, which
appears somewhere down the list due to the whims of news feeds, but I got
interrupted and forgot to address this issue.

If you've got

     pthread_mutex_t mutexA = PTHREAD_MUTEX_INITIALIZER;
     pthread_mutex_t mutexB = PTHREAD_MUTEX_INITIALIZER;

     char dataA;
     char dataB;

And one thread locks mutexA and writes dataA while another locks mutexB and
writes dataB, you risk word tearing, and incorrect results. That's a "platform
issue", that, as someone else commented, POSIX doesn't (and can't) address.

What do you do? I always advise that you keep a mutex and the data it protects
closely associated. As well as making the code easier to understand, it also
addresses problems like this. If the declarations were:

     typedef struct dataTag {
         pthread_mutex_t mutex;
         char data;
     } data_t;

     data_t dataA = {PTHREAD_MUTEX_INITIALIZER, 0};
     data_t dataB = {PTHREAD_MUTEX_INITIALIZER, 1};

You can now pretty much count on having the two data elements allocated in
separate "memory access chunks". Not an absolute guarantee, since a
pthread_mutex_t might be a char as well, and some C compilers might not align
structures on natural memory boundaries. But most compilers on machines that
care WILL align/pad structures to fit the natural data size, unless you
override it with a switch or pragma (which is generally a bad idea even when
it's possible). And, additionally, a pthread_mutex_t is unlikely to be less
than an int, and is likely at least a couple of longs. (On Digital UNIX, for
example, a pthread_mutex_t is 48 bytes, and on Solaris it's 24 bytes.)

There are, of course, no absolute guarantees. If you want to be safe and
portable, you might do well to have a config header that typedefs
"smallest_safe_data_unit_t" to whatever's appropriate for the platform. Then
it's just a quick trip to the hardware reference manual when you start a port.
On a CISC, you can probably use "char". On most RISC systems, you should use
"int" or "long".

Yes, this is one more complication to the process of threading old code. But
then, it's nothing compared to figuring out which data is shared and which is
private, and then getting the locking protocols right.

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/

> If I'm not mistaken, isn't that spelled:
>
>     #include 
>
>     typedef sig_atomic_t smallest_safe_data_unit_t;

You are not mistaken, and thank you very much for pointing that out. While I'd
been aware at some point of the existence of that type, it was far from the top
of my mind.

If you have data that you intend to share without explicit synchronization, you
should be safe in using sig_atomic_t. Additionally, using sig_atomic_t will
protect you against word tearing in adjacent data protected by separate mutexes.

There are additional performance considerations, such as "false sharing" effects
in cache systems, that might dictate larger separations between two shared pieces
of data: but those won't affect program CORRECTNESS, and are therefore more a
matter of tuning for optimal performance on some particular platform.

==============================================
 Q157: Mixing threaded/non-threadsafe shared libraries on Digital Unix  

claude vallee wrote:

> Hi All.  I have a question on building a mutli-threaded process (on
> Digital Unix 4.0) which is linked with non thread safe shared libraries.
>
> Let's say:
>
> mymain.c has no calls to thread functions and none of its functions runs
> in a secondary thread.  I will compile this file with the -pthread
> option. (I call secondary thread any but the main thread)
>
> liba.so contains non thread safe functions, but I know for a fact that
> none of its functions will run in a secondary thread.  This library was
> not built using the -pthread option.
>
> libb.so is my multi-thread library.  It creates threads and its
> functions are thread safe or thread reentrant.  All of its code was
> compiled with the -pthread option.  All the code executing in a
> secondary thread is in this library.
>
> The questions are:
>
> 1. Will this work?  If liba.so was not built with threads options, is it
> all right if it runs only in the main thread?  Which c runtime library
> will be used at run time? libc or libc_r?

On Digital UNIX 4.0D, this should be OK. On earlier versions, you need to
be careful around use of errno. For various historical reasons I won't try
to explain (much less justify), setting of errno through libc (using the
"seterrno()" function or equivalent) would previously result in setting the
process global errno ("extern int errno;"), not just the per-thread errno
of the calling thread.

For 4.0D, I was able to change the code so that seterrno() always sets the
calling thread's errno, and also sets the global errno ONLY if called from
the initial ("main") thread of the process. With this change, it's safe (as
least as far as errno use) to run non-thread-aware libraries as long as you
use them only in the initial thread.

To make this clear, prior to 4.0D, your liba code running in the main
thread may see errno change at random. As long as liba doesn't read errno,
this shouldn't be a problem.

You do have to be aware of whether liba SETS the global errno -- because
your thread-safe code won't see the global errno through any normal
mechanisms.

> 2. I noticed that on my DU 4.0, the libc.so and libc_r.so are
> identical!!  I assume this means that I am always using the thread safe
> version of the libc library.  Is that correct?

Yes -- libc_r was another historical oddity. (Due to OSF/1 rules.) It no
longer exists, as of Digital UNIX 4.0. The (hard) link provides binary
compatibility for older programs that still reference libc_r.

> 3. What does -pthread do to my code?  I saw that my objects are
> different (in size anyway), and that my executable point to libmach and
> libpthread.  What is added to the object code?

There are two basic contributions of "-pthread":

   * At compile-time, the definition -D_REENTRANT is provided
   * At link-time, the proper libraries are added, to the end of the actual
     list of libraries, but immediately before the implicit reference to
     libc that's generated by the compile driver. Additionally, -pthread
     causes the linker to search for "reentrant" versions of any library
     you specify. (E.g., if you say "-lfoo" and there's a libfoo_r in your
     -L path, the linker will automatically use it.)

The primary effect of -D_REENTRANT is to change  -- references to
errno make a call into the thread library to get the thread's private errno
address rather than the global errno. There are some other changes to
various headers, though.

> 4. Does defining _THREAD_SAFE when compiling and linking with
> libpthread, libmach and libc_r equivalent to building with the -pthread
> option?

No, _THREAD_SAFE doesn't do anything. It's considered obsolete. You should
use _REENTRANT. (Though I actually prefer the former term, I've never felt
it was worth arguing, or making anyone change a ton of header files.)

> I did some tests, and everything works well... for the moment, but IMHO,
> it does not mean anything.  Everyone knows that non thread safe code
> will work perfectly fine until your demo ;-)

Depends. If the demo is a critical requirement for a multi-million dollar
sale, then, yeah, it can probably hurt you worst by failing then.
Otherwise, though, it'll have a lot more fun by SUCCEEDING at the demo, and
failing when the customer runs the code in their mission-critical
environment. This is a correllary to a correllary to Murphy's Law, which
stated something about the inherent perversity of inanimate objects...

Oh... and since liba is, presumably, a third-party library over which
you've got no direct control... you should tell them immediately that
you're running their code in a threaded application, and it would be to
their long-term benefit to build a proper thread-safe version before you
find another option. If liba is YOUR code, then please don't play this
game: build it at least with -D_REENTRANT.

/---------------------------[ Dave Butenhof ]--------------------------\

==============================================
 Q158: VOLATILE instead of mutexes?  


> What about exception handlers ? I've always thought that when you had
> code like:
>
>         int i;
>
>         TRY
>         {
>                 
>                 . . .
>                 proc();
>         }
>         CATCH_ALL
>         {
>                 if (i > 0)
>                 {
>                         . . .
>                 }
>                 . . .
>         }
>
> that you needed to declare "i" to be volatile least the code in the
> catch block assume that "i" was stored in some register the contents
> of which were overwritten by the call to "proc" (and not restored by
> whatever mechanism was used to throw the exception).

Since neither ANSI C nor POSIX has any concept remotely resembling "exceptions", this
is all rather moot in the context of our general discussion, isn't it? I mean, it's
got nothing to do with sharing data between threads -- and that's what I thought we
were talking about. But sure, OK, let's digress.

Since there's no standard covering the behavior of anything that uses exceptions, (at
least, not if you use them from C, or even if you use the DCE exception syntax you've
chosen from C++), there's no portable behavior. Your fate is in the hands of the
whimsical (and hypothetically malicious ;-) ) implementation. This situation might
lead a cautious programmer to be unusually careful when treading in these waters, and
to wear boots with thick soles. (One might also say that it could depend on exactly
HOW you "foodle with i", but I'll choose to disregard an entire spectrum of mostly
amusing digressions down that fork.)

Should you use volatile in this case? Sure, why not? It might not be necessary on
many platforms. It might destroy your performance on any platform. And, where it is
necessary, it might not do what you want. But yeah, what the heck -- use it anyway.
It's more likely (by some small margin) to save you than kill you.

Or, even better... don't code cases like this!

/---------------------------[ Dave Butenhof ]--------------------------\
==============================================
 Q159: After pthread_cancel() destructors for local object do not get called?!  
 
> Hello,
> I've run into a trouble when I found out that when I cancel a thread via
> pthread_cancel() than destructors for local object do not get called.
> Surprising :). But how to deal with this? With a simple  thread code
> it would not be a big problem, but in my case it's fairly complex code,
> quite a few STL classes etc. Has someone dealt with such problem and is
> willing to share his/her soltution with me ? I thought I could 'cancel'
> thread via pthread_kill() and raise an exception within a signal handler
> but it's probably NOT very good idead, is it?;)
> Thank you,
>         Ales Pour
> 
> Linux, egcs-1.0.3, glibc-2.0.7 with LinuxThreads
Ales,

  Unfortunately, not surprising.  C++ has not formally decided what to do with
thread cancellation, so it becomes compiler-specific.  The Sun compiler (for 
example) will run local object destructors upon pthread_exit() (hence 
cancellation also).  Others may not.

  I suppose the best GENERAL C++ solution is:

	a) Don't use stack-allocated objects.
	b) Don't use cancellation.

  Otherwise you can simply insist on a C++ compiler that runs the destructors.

-Bil
==============================================
 Q160: No pthread_exit() in Java.  

 >    In POSIX, we have pthread_exit() to exit a thread.  In Java we
 >  *had* Thread.stop(), but now that's gone.  Q: What's the best way
 >  to accomplish this?
 >  
 >    I can (a) arrange for all the functions on the call stack to
 >  return, all the way up to the top, finally returning from the
 >  top-level function.  I can (b) throw some special exception I
 >  build for the purpose, TimeForThreadToExitException, up to the
 >  top-level function.  I can throw ThreadDeath.
 >  
 >    But what I really want is thread.exit().
 >  
 >    Thoughts?
 >  
 >  -Bil
 > -- 
 > ================
 > Bil@LambdaCS.com
 > 
 > http://www.LambdaCS.com
 > Lambda Computer Science 
 > 555 Bryant St. #194 
 > Palo Alto, CA,
 > 94301 
 > 
 > Phone/FAX: (650) 328-8952
 > 

Here's a real quick reply (from a slow connecction from
Sydney AU (yes, visiting David among other things)). I'll
send something more thorough later....

Throwing ThreadDeath yourself is a pretty good way to force current
thread to exit if you are sure it is in a state where it makes sense
to do this.

But if you mean, how to stop other threads: This is one reason why
they are extremely unlikely to actually remove Thread.stop(). The next
best thing to do is to take some action that is guaranteed to cause
the thread to hit a runtime exception. Possibililies range from the
well-reasoned -- write a special SecurityManager that denies all
resource-checked actions, to the sleazy -- like nulling out a pointer
or closing a stream that you know thread needs. See
  http://gee.cs.oswego.edu/dl/cpj/cancel.html
for a discussion of some other alternatives.


-Doug


ThreadDeath is an Error (not a checked Exception, since app's routinely
catch all checked Exceptions) which has just the semantics you are talking
about: it is a Throwable that means "this thread should die".  If
you catch it (because you have cleanup to do), you are SUPPOSED to
rethrow it.  1.2 only, though, I think.  Thread.stop() uses it, but
although stop() is deprecated, it appears that ThreadDeath is not.

I think.  :^)

Nicholas

There is *nothing* special about a ThreadDeath object. It does not mean
"this thread should die" but rather it indicates that "this thread has
been asked to die". The only reason it "should" be rethrown is that if
you don't then the thread doesn't actually terminate. This has always
been documented as such and is not specific to 1.2.

If a thread decides that for some reason it can continue with its work
then it can simply throw  new ThreadDeath() rather than calling stop()
on itself. The only difference is that with stop() the Thread is
immediately marked as no longer alive - which is a bug in itself.

Cheers,
David
==============================================
 Q161: Is there anyway I can make my stacks red zone protected?  

Allocate your stack segments using mmap.  Use mprotect to make the
page after the bottom of your stack read-only (I'm assuming the stack
grows down on whatever system you're using), or leave a hole in your
address space.  If you get a segfault due to an attempted write at the
top of a red zone, map in some more stack and build a new red zone.



==============================================
 Q162: Cache Architectures, Word Tearing, and VOLATILE


Tim Beckmann wrote:

> Dave Butenhof wrote:
> > > David,
> > >
> > > My thoughts exactly!
> > >
> > > Does anyone know of a mainstream architecture that does this sort of
> > > thing?
> >
> > Oh, absolutely. SPARC, MIPS, and Alpha, for starters. I'll bet most other RISC
> > systems do it, too, because it substantially simplifies the memory subsystem
> > logic. And, after all, the whole point of RISC is that simplicity means speed.
>
> MIPS I know :)  The latest MIPS processors R10K and R5K are byte addressable.
> The whole point of RISC is simplicity of hardware, but if it makes the software
> more complex it isn't worth it :)

The whole idea of RISC is *exactly* to make software more complex. That is,
by simplifying the hardware, hardware designers can produce more stable
designs that can be produced more quickly and with more advanced technology
to result in faster hardware. The cost of this is more complicated
software. Most of the complexity is hidden by the compiler -- but you can't
necessarily hide everything. Remember that POSIX took advantage of some
loopholes in the ANSI C specification around external calls to proclaim that
you can do threaded programming in C without requiring expensive and awkward
hacks like "volatile". Still, the interpretation of ANSI C semantics is
stretched to the limit. The situation would be far better if a future
version of ANSI C (and C++) *did* explicitly recognize the requirements of
threaded programming.

> > If you stick to int or long, you'll probably be safe. If you use anything
> > smaller, be sure they're not allocated next to each other unless they're under
> > the same lock.
>
> Actually, you can be pretty sure that a compiler will split two declarations
> like:
>         char dataA;
>         char dataB;
> to be in two separate natural machine words.  It is much faster and easier for
> those RISC processors to digest.  However if you declare something as:

While that's certainly possible, that's just a compiler optimization
strategy. You shouldn't rely on it unless you know FOR SURE that YOUR
compiler does this.

>         char data[2]; /* or more than 2 */
> you have to be VERY concerned with the effects of word tearing since the
> compiler will certainly pack them into a single word.

Yes, this packing is required. You've declared an array of "char" sized
data, so each array element had better be allocated exactly 1 char.

> > I wrote a long post on most of the issues brought up in this thread, which
> > appears somewhere down the list due to the whims of news feeds, but I got
> > interrupted and forgot to address this issue.
>
> Yep, I saw it.  It was helpful.  So was the later post by someone else who
> included a link to a DEC alpha document that explained what a memory barrier
> was in this context.  I've seen three different definitions over the years.
> The definition you described in your previous post agreed with the DEC alpha
> description... That a memory barrier basically doesn't allow out of order
> memory accesses to cross the barrier.  A very important issue if you are
> implementing mutexes or semaphores :)[...]
>
> However, I really believe that dataA and dataB should both be declared as
> "volatile" to prevent the compiler from being too aggressive on it's
> optimization.  The mutex still doesn't guarantee that the compiler hasn't
> cached the data in an internal register across a function call.  My memory
> isn't perfect, but I do think this bit me on IRIX.

The existence of the mutex doesn't require this, but the semantics of POSIX
and ANSI C do require it. Remember that you lock a mutex by calling a
function, passing an address. While an extraordinarily aggressive C compiler
with a global analyzer might be able to determine reliably that there's no
way that call could access the data you're trying to protect, such a
compiler is unlikely -- and, if it existed, it would simply violate POSIX
1003.1-1996, failing to support threads.

You do NOT need volatile for threaded programming. You do need it when you
share data between "main code" and signal handlers, or when sharing hardware
registers with a device. In certain restricted situations, it MIGHT help
when sharing unsynchronized data between threads (but don't count on it --
the semantics of "volatile" are too fuzzy). If you need volatile to share
data, protected by POSIX synchronization objects, between threads, then your
implementation is busted.

> > There are, of course, no absolute guarantees. If you want to be safe and
> > portable, you might do well to have a config header that typedefs
> > "smallest_safe_data_unit_t" to whatever's appropriate for the platform. Then
> > it's just a quick trip to the hardware reference manual when you start a port.
> > On a CISC, you can probably use "char". On most RISC systems, you should use
> > "int" or "long".
>
> There never are guarantees are there :)

To reiterate again one more time, ( ;-) ), the correct (ANSI C) portable
type for atomic access is sig_atomic_t.

> > Yes, this is one more complication to the process of threading old code. But
> > then, it's nothing compared to figuring out which data is shared and which is
> > private, and then getting the locking protocols right.
>
> But what fun would it be if it wasn't a challenge :)

Well, yeah. That's my definition of "fun". But not everyone's. Sometimes
"boring and predictable" can be quite comforting.

> However, I would like to revist the original topic of whether it is "safe" to
> change a single byte without a mutex.  Although, instead of "byte" I'd like to
> say "natural machine word" to eliminate the word tearing and non-atomic memory
> access concerns.  I'm not sure it's safe to go back to the original topic, but
> what the heck ;)

sig_atomic_t.

> If you stick to a "natural machine word" that is declared as "volatile",
> you do not absolutely need a mutex (in fact I've done it).  Of course, there are
> only certain cases where this works and shouldn't be done unless you really know
> your hardware architecture and what you're doing!  If you have a machine with a
> lot of processors, unnecessarily locking mutexes can really kill parallelism.
>
> I'll give one example where this might be used:
>
> volatile int stop_flag = 0;  /* assuming an int is atomic */
>
> thread_1
> {
>         /* bunch of code */
>
>         if some condition exists such that we wish to stop thread_2
>                 stop_flag = 1;
>
>         /* more code - or not :) */
> }
>
> thread_2
> {
>         while(1)
>         {
>                 /* check if thread should stop */
>                 if (stop_flag)
>                         break;
>
>                 /* do whatever is going on in this loop */
>         }
> }
>
> Of course, this assumes the hardware has some sort of cache coherency
> mechanism.  But I don't believe POSIX mutex's or memory barriers (as
> defined for the DEC alpha) have any impact on cache coherency.

If a machine has a cache, and has no mechanism for cache coherency, then it
can't work as a multiprocessor.

> The example is simplistic, but it should work on a vast majority of
> systems.  In fact the stop_flag could just as easily be a counter
> of some sort as long as only one thread is modifying the counter...

In some cases, yes, you can do this. But, especially with your "stop_flag",
remember that, if you fail to use a mutex (or other POSIX-guaranteed memory
coherence operation), a thread seeing stop_flag set CANNOT assume anything
about other program state. Nor can you ensure that any thread will see the
changed value of stop_flag in any particular bounded time -- because you've
done nothing to ensure memory ordering, or coherency.

And remember very carefully that bit about "as long as only one thread is
modifying". You cannot assume that "volatile" will ever help you if two
threads might modify the counter at the same time. On a RISC machine,
"modify" still means load, modify, and store, and that's not atomic. You
need special instructions to protect atomicity across that sequence (e.g.,
load-lock/store-conditional, or compare-and-swap).

Am I trying to scare you? Yeah, sure, why not? If you really feel the need
to do something like this, do yourself (and your project) the courtesy of
being EXTREMELY frightened about it. Document it in extreme and deadly
detail, and write that documentation as if you were competing with Stephen
King for "best horror story of the year". I mean to the point that if
someone takes over the project from you, and doesn't COMPLETELY understand
the implications, they'll be so terrified of the risk that they'll rip out
your optimizations and use real synchronization. Because this is just too
dangerous to use without full understanding.

There are ways to ensure memory ordering and coherency without using any POSIX
synchronization mechanisms, on any machine that's capable of supporting POSIX
semantics. It's just that you need to be really, really careful, and you need to be
aware that you're writing extremely machine-specific (and therefore inherently
non-portable) code. Some of this is "more portable" than others, but even the
"fairly portable" variants (like your stop_flag) are subject to a wide range of
risks. You need to be aware of them, and willing to accept them. Those who aren't
willing to accept those risks, or don't feel inclined to study and fully understand
the implications of each new platform to which they might wish to port, should
stick with mutexes.

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/


==============================================
 Q163: Can ps display thread names?


>  Is there a way to display the name of a thread (specified thanks
>  to the function pthread_setname_np) in commands such a ps ?
>  (in order to quickly see the behavior of a well-known thread).
>  If it is not possible with Digital UNIX's ps, someone may
>  have hacked some interesting similar utilities that display
>  such thread informations ?

The ps command is a utility to show system information, and this would be
getting into an entirely different level of process information. It would,
arguably, be "inappropriate" to do this in ps. In any case, the decision
was made long ago to not do as you suggest.

The easiest way to get this information is to attach to the process with
ladebug (or another suitable user-level-thread-enabled debugger) and ask
for the information. (E.g., ladebug's "show thread" command.)

While one certainly could create a standalone utility, you'd need to find
the binary associated with the process, look up symbols, use /proc (or
equivalent) to access process memory, and so forth -- sounds a lot like a
debugger, doesn't it?

The mechanism used to access this information is in libpthreaddebug.so. As
of 4.0D, the associated header file,  is available on the
standard OS kit (with the other development header files). Although it's
not externally documented, it's heavily commented, and reasonably
self-describing.

==============================================
 Q164: (Not!) Blocking on select() in user-space pthreads.

Subject: Re: Blocking on select() in user-space pthreads under HP/UX 10.20 

David Woodhouse wrote:

> HP/UX 10.20 pthreads as are implemented as user-space as opposed to
> kernel. I've heard rumors that a user-space thread that blocks on
> select() actually blocks all other threads within that process (ie the
> entire process). True of false?

The answer is an absolute, definite, unambigous... maybe.

Or, to put it another way... the answer is true AND false.

However, being in a generous (though slightly offbeat) mood today, I'll go a
little further and explain the answer. (Remember how the mice built Deep
Thought to find the answer to "Life, the Universe, and Everything", and it
came back with the answer "42", so they had to go off and build an entirely
different computer to find the question, which was "what is 9 times 6",
resulting in a third attempt, this time to find out what the question and
answer MEANT?)

Anyway, any blocking kernel call, including select, will indeed block the
process. However, if built correctly, a DCE thread (that's the origin of the
thread library on 10.20) application will never directly call select.
Instead, its calls will be redirected to the user-mode thread library's
"thread-synchronous I/O" package. This package will attempt a NON-blocking
select, and, if it would have needed to block (none of the files are
currently ready), the thread will be blocked on a condition variable "until
further notice". At various points, the library polls (with select) and
awakens any thread waiting for a file that's now ready. When all application
threads are blocked somewhere, the thread library blocks the process in
select, with a mask that OR-s all the masks for which any thread is waiting,
and with a timeout representing the next user-mode timer
(pthread_cond_timewait, select, whatever).

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/


==============================================
 Q165: Getting functional tests for UNIX98 

> Dave Butenhof wrote somewhere that there were
> a set of functional tests for UNIX98, that could
> also work with POSIX. Any idea where I could find
> it?

The place to look is The Open Group. Start with http://www.xopen.org/.
(Unfortunately I don't have a bookmark handy for the test suite, and I
can't get to xopen.org right now; so you're on your own from here. ;-))

==============================================
 Q166: To make gdb work with linuxthreads?  

Are there any ongoing work or plans to make gdb work with linuxthreads?
>
>- Erik

Yes, there's a project at the University of Kansas called SmartGDB that is 
working on support for user-level and kernel-level threads.  There is 
already support for several user level thread packages and work is 
currently being done on the linuxthreads support.  The URL is:
    http://hegel.ittc.ukans.edu/projects/smartgdb/

We have most of the kernel modifications done required to support it and 
are working on the rest of the changes to gdb.  At this point, I can't 
even guess on a release date, but you can check the web page for more 
information on what's been done so far.  The email contact is 
smartgdb@ittc.ukans.edu.

Robert
==============================================
 Q167: Using cancellation is *very* difficult to do right...  

Bil Lewis wrote:

> Dave Butenhof wrote:
> > >   Using cancellation is *very* difficult to do right, and you
> > > probably don't want to use it if there is any other way you can
> > > accomplish your goal.  (Such as looking at a "finish" flag as you
> > > do below.)
> >
> > I don't agree that cancellation is "very" difficult, but it does
> > require understanding of the application, and some programming
> > discipline. You have to watch out for cancellation points, and be
> > sure that you've got a cleanup handler to restore shared data
> > invariants and release resources that would otherwise be stranded if
> > the thread "died" at that point. It's no worse than being sure you
> > free temporary heap storage, or unlock your mutexes, before
> > returning from a routine... but that's not to say that it's trivial
> > or automatic. (And I'm sure we've never gotten any of those things
> > wrong... ;-) )
>
>   Dave has written 10x as much UNIX code as I have, so our definitions
> of "very difficult" are distinct.  (I've probably been writing MP code
> longer tho...  I built my first parallel processor using two PDP/8s
> back in '72.  Now THERE was a RISC machine!  DEC could have owned the
> world if they'd tried.  I remember when...)

Yeah, PDP-8 was a pretty good RISC, for the time. Of course it needed
virtual memory, and 12 bits would now be considered a rather "quirky"
word size. But, yeah, those could have been fixed.

Oh yeah... and we DID own the world. We just let it slip out of our
hands because we just laughed when little upstarts said they owned it.
(Unfortunately, people listened, and believed them, and eventually it
came to be true.) ;-) ;-)

>   It's that bit "to restore shared data invariants". Sybase, Informix,
> Oracle, etc. spend years trying to get this right.  And they don't
> always succeed.

It's hard to do hard things. Yeah, when you've got an extremely
complicated and extremely large application, bookkeeping gets more
difficult. This applies to everything, not just handling cancellation.
Just as running a multinational corporation is harder than running a
one-person home office. The point is: the fact that the big job is hard
doesn't mean the small job is hard. Or, you get out what you put in. Or
"thermodynamics works". Or whatever.

>   And don't forget to wait for the dead threads.  You can't do
> anything with the shared data until those have all been joined,
> because you can't be sure when they actually die.

That's untrue, as long as you use proper synchronization (or maybe "as
long as you use synchronization properly"). That's exactly why the mutex
associated with a condition wait is re-locked even when the wait is
cancelled. Cleanup code needs (in general) to restore invariants before
the mutex can be safely unlocked. (Note that while the data must be
"consistent", at the time of the wait, because waiting has released the
mutex, it's quite common to modify shared data in some way associated
with the wait, for example, adding an item to a queue; and often that
work must be undone if the wait terminates.)

You only need to wait for the cancelled thread if you care about it's
return value (not very interesting in this case, since it's always
PTHREAD_CANCELED, no matter how many times you look), or if you really
want to know that it's DONE cleaning up (not merely that the shared data
is "consistent", but that it conforms to some specific consistency -- an
attempt that I would find suspicious at best, at least if there might be
more than the two threads wandering about), or if you haven't detached
the thread and want to be sure it's "gone".

>   Conclusion: Dave is right (of course).  The definition of "very" is
>   up for grabs.

The definition of the word "very" is always up for grabs. As Samuel
Clemens once wrote, when you're inclined to use the word "very", write
"damn" instead; your editor will remove it, and the result will be
correct.

Sure, correct handling of cancellation doesn't happen automatically.
Neither does correct use of threads, much less correct use of the arcane
C language (and if C is "arcane", what about English!?) Somehow, we
survive all this.

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/

==============================================
 Q168: Why do pthreads implementations differ in error conditions?  
 

taylor@template.com wrote:

> I'd like to understand why pthreads implementations from different
> vendors define error conditions differently.  For example, if
> pthread_mutex_unlock is called for a mutex that is not owned by the
> calling thread.
>
>    Under Solaris 2.5:  "If the calling thread is not the owner of the
>    lock, no error status is returned, and the behavior of the program
>    is undefined."
>
>    Under AIX 4.2:  It returns the EPERM error code.
>
> The problem may be that the AIX 4.2 implementation is based on draft 7
> of the pthreads spec, not the standard, but I certainly prefer the AIX
> approach.
>
> Another example:  pthread_mutex_lock is called for a mutex that is
> already owned by the calling thread.
>
>    Under Solaris 2.5: "If the current owner of a mutex tries to relock
>    the mutex, it will result in deadlock." (The process hangs.)
>
>    Under AIX 4.2: It returns the EDEADLK error code.
>
> Once again, the AIX approach certainly seems preferable.
>
> Aren't these issues clearly defined by the pthreads standard?  If not,
> why not?

Yes, these issues are clearly defined by the POSIX standard. And it's
clearly defined in such a manner that implementations are not required
to make the (possibly expensive) checks to report this sort of
programmer error -- but so that implementations that do choose to detect
and report the error must do so using a standard error code.

In this case, Solaris 2.5 chooses not to "waste" the time it would take
to detect and report your error, while AIX 4.2 does. Both are correct
and conform to the standard. (Although, as you pointed out, AIX 4.2
implenents an obsolete draft of the standard, in this respect it doesn't
differ substantially from the standard.)

The POSIX philosophy is that errors that are not under programmer
control MUST (or, in POSIX terms, "SHALL") be reported. Examples include
ENOMEM, and other resource shortages. You can't reasonably know that
there's not enough memory to create a thread, because you can't really
know how much you're using, or how much additional is required. On the
other hand, you can be expected to know that you've already locked the
mutex, and shouldn't try to lock it again. POSIX does not require that
an implementation deal gracefully with such programmer errors.

While it is nice to have a "debugging mode" where all programmer errors
are detected, in the long run it's more important to have a "production
mode" where such extremely critical functions as mutex lock and unlock
execute as quickly as possible. In general, the only way to do both is
to have two separate libraries. This complicates maintenance
substantially, of course -- but it also complicates application
development because the two libraries will have different timings, and
will expose different problems in the application design. Which means
you'll inevitably run into a case that only fails on the "production
library", and can't be debugged using the "debug library". That usually
means the development and maintenance costs of building and shipping two
thread libraries usually isn't worthwhile.

You're better off relying on other tools to detect this class of
programming error. For example, the Solaris "lock_lint" program, or
dynamic trace analysis tools that can check for incorrect usage at
runtime in your real program.

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/

==============================================
 Q169: Mixing threaded/non-threadsafe shared libraries on DU  
 
Claude Vallee wrote:

> Thanks Dave Butenhof for your excellent answer.  I just have a few
> complementary questions.
>
> + To make this clear, prior to 4.0D, your liba code running in the
> + main thread may see errno change at random. As long as liba doesn't
> + read errno, this shouldn't be a problem.
> +
>
> I found out I was using 4.0B.  Is errno the only the problem area of
> the c run time library?  What about other libraries like librt?

The base kit libraries should be thread-safe. There are so many, though,
that I'm afraid I can't claim personal knowledge that they all ARE
thread-safe. I also know of at least one case where a base kit library
was "almost" thread-safe, but had been compiled without -D_REENTRANT
(making it subject to errno confusion). Bugs, unfortunately, are always
possible. While it's not that hard to code a library for thread-safety,
it's nearly impossible to PROVE thread-safety -- because problems come
in the form of race conditions that are difficult to predict or provoke.

> + You do have to be aware of whether liba SETS the global errno --
> + because your thread-safe code won't see the global errno through any
> + normal mechanisms.
>
> What do you mean by that?  Yes, liba sets errno each time it calls a
> system service (the system service sets it actually).  If you're
> asking if it explicitely sets it, then no.  Are you asking if I am
> counting on setting errno from one thread and reading it from the
> other thread counting on the value to be the same?

Calling system services doesn't count. The libc syscall stubs that
actually make the kernel call DO handle errno correctly with threads.
(On the other hand, if your library runs in a threaded application and
isn't built correctly, you'll end up looking at the global errno while
the system/libc just set your thread errno. That's the point of my 4.0D
change -- at least if that non-thread-safe code runs in the initial
thread, it'll get the right errno.)

> By the way, seterrno(), does not seem to be a public service (it
> doesn't have a man page anyway, (I found _Seterrno() in errno.h, but I
> we're certainly not using it )).

You're right -- int _Geterrno() and _Seterrno(int) are the external
interfaces. I'd recommend compiling for the threaded errno rather than
using those calls, though.

> + No, _THREAD_SAFE doesn't do anything. It's considered obsolete. You
> + should use _REENTRANT. (Though I actually prefer the former term,
> + I've never felt it was worth arguing, or making anyone change a ton
> + of header files.)
>
> Ok, _THREAD_SAFE is out.  Then, if I define _REENTRANT when compiling
> all my sources, and I explicitely link with libpthread, libmach,
> libc_r, and all the reentrant versions of my libraries, will this
> achieve the same thing as using the "-pthread" option?  (Or am I
> playing with fire again?).

We document the equivalents, and it's perfectly legal to use them.
However, the actual list of libraries may change from time to time, and
you'll get the appropriate set for the current version, automatically,
when you link with "cc -pthread". Over time, using the older set of
libraries may leave you carrying around extra baggage. For example, your
reference to libc_r is long-since obsolete; and on 4.0D, threaded
applications no longer need libmach. While -pthread links automatically
stop pulling in these useless references, you still be carrying them
around, costing you extra time at each program load, as well as extra
virtual memory. Is it a big deal? That's up to you. But if you're using
a compiler that supports "-pthread", I'd recommend using it.

> + If liba is YOUR code, then please don't play this
> + game: build it at least with -D_REENTRANT.
>
> Yes, liba is our code... Actually, liba is a set of hundreds of
> libraries which take a weekend to build.  And most of our processes
> are not multithread.  What I was trying to achieve is to save on
> processing time (use non thread-safe libraries in single threaded
> processes), and to save on compile time (not building both single
> threaded and multi threaded versions of all the libraries).

If you just compile with -D_REENTRANT, you'll get thread-safe errno, but
that's only a small part of "thread safety". As long as it's only called
in one thread, that's probably OK. For more general thread-safety, with
very little performance impact on non-threaded processes, you might look
into the TIS interfaces (tis_mutex_lock, tis_mutex_unlock, etc.). You
can use these just the equivalent POSIX functions; in a threaded
process, they do the expected synchronization, but in a non-threaded
process they avoid the cost of interlocked data access and memory
barrier instructions, giving very low overhead. (TIS is better than
having pthread stubs in libc, because it works even when the main
program isn't linked against libpthread.)

> Thanks again for a thorough answer.  By the way, for some reason, I
> could never get the answer to my question from my news server (I got
> it from searching usenet through altavista), so please reply by email
> as well as through the newsgroup.

Yeah, news servers can be annoying creatures, with their own strange
ideas of what information you should be allowed to see. You really can't
trust them! ;-)

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/

==============================================
 Q170: sem_wait() and EINTR  

W. Richard Stevens wrote:

> Posix.1 says that sem_wait() can return EINTR.  Tests on both Solaris 2.6
> and Digital Unix 4.0b show that both implementations do return EINTR when
> sem_wait() is interrupted by a caught signal.  But if you look at the
> (albeit simplistic) implementation of semaphores in the Posix Rationale
> that uses mutexes and condition variables (pp. 517-520) sem_wait() is
> basically:
>
>     int
>     sem_wait(sem_t *sem)
>     {
>             pthread_mutex_lock(&sem->mutex);
>             while (sem->count == 0)
>                     pthread_cond_wait(&sem->cond, &sem->mutex);
>             sem->count--;
>             pthread_mutex_unlock(&sem->mutex);
>             return(0);
>     }
>
> But pthread_cond_wait() does not return EINTR so this function will never
> return EINTR.  So I was wondering how existing implementations actually
> implement sem_wait() to detect EINTR.  Seems like it would be a mess ...
>
>         Rich Stevens


On Digital UNIX, sem_wait() turns into a system call, with the usual behavior
in regard to signals and EINTR. You can only implement POSIX semaphores via
pthreads interfaces if you have support for the PSHARED_PROCESS synch
attribute. Digital
UNIX won't have that support until an upcoming major release.
At that point, it's entirely possible that the POSIX semaphores
will be re-implemented to use the model you cite. I certainly
encouraged the maintainer to do so before I left...

__________________________________________________
Jeff Denham (jdenham@brighttiger.com)

Bright Tiger Technologies
SmartCluster™ Software for Distributed Web Servers
http://www.brighttiger.com

==============================================
 Q171: pthreads and sprocs  

Peter Shenkin wrote:
> 
> Can someone out there say something about which is better to use
> for new code -- pthreads or sprocs -- and about what the tradeoffs
> are?

A good question.  I've done little with sprocs, but have used pthreads
on a 32p SGI Origin a good deal.

Sprocs are heavyweight processes; pthreads are MUCH lighter weight.

Sprocs have a considerable amount of O/S and programmer tool support 
for high performance programming; as yet, pthreads has almost none.

Sprocs lend themselves to good fine-grain control of resources
(memory, CPU choice, etc); as yet these strengths are largely lacking 
in SGI pthreads.

The project on which I work has bet the farm on the present and future 
high performance of pthreads and the results so far have been good.
However, we would dearly love for SGI and the rest of the parallel
programming community to better support pthreads as well as they have 
their former proprietary parallel programming models so that we can
control our threads as specifically as we could our sprocs and the like.

Not really a complaint; more of a strong request.

The upshot is, IMHO, go ahead and program in pthreads on SGIs.  The
performance gain you would have gotten from better control of your
sprocs is made up for in the "portability" of pthreads, the rosier
future of pthreads, and their more modest system resource use.

Not my company's official position, I should add.

--
Randy Crawford
rcrawford@erols.com


==============================================
 Q172: Why are Win32 threads so odd?  

Bil Lewis  wrote in article
<351B67B5.50DB@LambdaCS.com>...
>   You must know the folks at MS who did Win32 threads (I assume).

Bad assumption. I know of them, but don't know them and they don't know me.

> Some of the design sounds so inefficient and awkward to use, while
> other bits look really nice. 

My own opinion is that win32 grew from uni-thread to pseudo-multi-thread in
a very haphazard manner, basically features were added when they were found
to be needed.
I personally dislike the overall asymmetric properties of the API. Consider
the current problem of providing POSIX condition variables: if you could
release a single waiter on a manual reset event, or release all waiters on
an autoreset event then the problem would be much simpler to solve.
Consider also the ability to signalObjectAndWait() but no corresponding
signalObjectAndwaitMultiple() - another change that would make writing
various forms of CV's easier.

>   Are my views widely shared in the MS world?  And why did they
> choose that design?  Have they thought of adopting the simpler
> POSIX design (if not POSIX itself)?

No idea, sorry. Try asking Dave Cutler he was one of the main thread
architects AFAIK.

David

Bil Lewis wrote:

>   The real question is: "What the heck was Cutler (I assume?) thinking when he made
> Win32 mutexes kernel objects?"
>

Well, Dave has a history in this department. Consider his
Digital VAXELN realtime executive, a direct predecessor of
WNT. It was designed and written by Cutler (and some other
folks who later contributed to NT, like Darryl Havens)
to run on the MicroVAX I waaaaay back in the early '80s
at DECWest in Seattle. (Development soon moved to a
dedicated realtime group at the Mill in Maynard.)

VAXELN had processes (called jobs) and threads (called
processes) and kernel objects (PROCESS, DEVICE, SEMAPHORE,
AREA, PORT, MESSAGE). It ran memory-resident on embedded
VAXes(or VAX 9000s for that matter), and let you program
device drivers (or whatever) in Pascal, C, FORTRAN,
or Bliss even. (Pretty nifty little concurrent environment,
a little bit too ahead of its time for its own good, guess.)

The only synchronization object provided oringally was the
semaphore, which like the NT mutex, required a trip into
the kernel even for uncontested locking. This of course
proved to be too expensive for real-world concurrent
programming, so a library-based optimized synch. object
was developed.

It had a count and an embedded binary semaphore object
that could be locked quickly in user space through the
use of the VAX ADAWI interlocked-increment instruction.
A system call occurred only for blocking and wakeup on a
contested lock.

Sounds just like an NT critical section, huh?

Ironically, in VAXELN it was called a MUTEX! History
repeats itself, with only the names changed to protect
the guilty...

Jeff

==============================================
 Q173: What's the point of all the fancy 2-level scheduling??  
saroj@bear.com wrote:

> In article <3509A1E5.4B76F2E6@brighttiger.com>,
>   Jeff Denham  wrote:
> >
> > Boris Goldberg wrote:
> >
>
> > Seriously, I've been around this Solaris threads package long enough to
> > be wondering how often anyone is using PROCESS scope threads.
> > With everyone just automatically setting SYSTEM scope threads
> > to get the expected behavior, what's the point of all the fancy 2-level
> > scheduling??
>
> I think, it is better to use thr_setconcurrency to create #of processors
> + some additional number (for I/O bound threads) of LWPs rather than
> creating LWP for each thread. Can Dave Butenhof or somebody from
> Sun thread designer team please comment on this?

That's kinda funny, since I've got no connection with Sun.

The real problem is that Sun's "2 level scheduling" really isn't at all like
2-level scheduling should be, or was intended to be. There's a famous paper
from the University of Washington on "Scheduler Activations" (one of Jeff
Denham's replies to this thread mentioned that term, so you may have noticed
it), which provides the theoretical basis for modern attempts at "2 level
scheduling". Both Sun and Digital, for example, claim this paper as the
basis for our respective 2-level scheduling models.

However, while we (that's Digital, not Sun) began with a model of 2-way
communication between kernel and user schedulers that closely approximates
the intended BEHAVIOR (though not the detailed implementation) of scheduler
activations, I have a hard time seeing anything usefully similar in Solaris.
They have a signal to save the process from total starvation when the final
thread blocks in the kernel (by giving the thread library a chance to create
another LWP). We automatically generate a new "replacement VP" so that the
process always retains the maximum level of concurrency to which its
entitled.

The advantages of 2-level scheduling are in performance and scaling.

  1. Scaling. Kernel threads are kernel resources, and (as with processes),
     there are strict limits to how many a kernel can handle. The limits are
     almost always fixed by some configuration process, and additionally
     limited by user quotas. Why? Because they're expensive -- not just to
     the process that uses them, but to the system as a whole. User-mode
     threads, on the other hand, are "just" virtual memory, and, in
     comparison, "dirt cheap". So you can create a lot more user threads
     than kernel threads. Yeah, the user threads can't all run at the same
     time... but neither can the kernel threads, because the number of
     processors (being a relatively expensive HARDWARE resource) is even
     more limited. The point is to balance the number of kernel threads
     against the "potentially parallelism" of the system (e.g., the number
     of processors), while balancing the number of user threads against the
     "potential concurrency" of the process (the maximum parallelism plus
     the maximum number of outstanding I/O operations the process might be
     able to undertake). [On Solaris, you do this manually by creating LWPs
     -- either by creating BOUND (SCS) threads, or by calling
     thr_setconcurrency. On Digital UNIX, this is done for you automatically
     through the integration between user and kernel level schedulers.]
  2. Performance. In many typical workloads, most of the communication is
     between threads within the process. Synchronization involves mutexes
     and condition variables. A 2-level scheduler can optimize these types
     of synchronization, and the resulting context switches, without
     involving the kernel at all. A kernel thread needs to call into the
     kernel to block -- and then another thread needs to call into the
     kernel again to unblock it. A user thread (or a 2-level thread blocking
     in user mode) only needs to call into the thread library. Because a
     call into the kernel is more expensive than a call within the process
     (and usually LOTS more expensive), this can save a lot of time over the
     life of a process.

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/
==============================================
 Q174: Using the 2-level model, efficency considerations, thread-per-X  
 
Bil Lewis wrote:

> My experience (which is fairly broad), and my opinion (which is
> extensive) is that THE way to do this is this:

Yes, Bil, your experience is extensive -- but unfortunately mostly limited
to Solaris, which has a poor 2-level scheduling design. (Sorry, guys, but
it's the truth.)

>   Use only System scoped threads everywhere.  On your selected
> release platforms, tune the number of threads for each configuration
> that's important.  (Take educated guesses at the rest.)  For servers,
> do everything as a producer/consumer model.  Forget Thread-per-request,
> thread-per-client, etc.  They are too hard to tune well, and too
> complex.

No, use SCS ("system scoped") threads only where absolutely necessary. But,
yeah, in recognition of your extensive experience, I would acknowledge that
on Solaris they are (currently) almost always necessary. (Solaris 2.6 is
supposed to be better than 2.5, though I haven't been able to try it, and I
know that Solaris developers have hopes to make substantial improvements in
the future.) This is, however, a Solaris oddity, and not inherent in the
actual differences between the PCS ("process scoped") and SCS scheduling
models.

On the rest, though -- I agree with Bil that you should avoid "thread per
request" in almost all cases. Although it seems like a simple extension
into threads, you usually won't get what you want. This is especially true
if your application relies on any form of "fairness" in managing the
inevitable contention between clients, because "thread per request" will
not behave fairly. You'll be tempted to blame the implementation when you
discover this, but you'll be wrong -- the problem is in the application.
The best solution is to switch to a client/server (or "producer/consumer")
model, where you control the allocation and flow of resources directly.

>   Process scoped threads are good for a very small number of unusual
> examples (and even there I'm not totally convinced.)

On the contrary, PCS threads are best except for the very few applications
where cross-process realtime scheduling is essential to correct operation
of the application. (E.g., for direct hardware access.)

>   Simplicity rules.

Right. (Fully recognizing the irony of agreeing with a simplistic statement
while disagreeing with most of the philosophy behind it.)

>   Logic:  Process scope gives some nice logical advantages in design,
> but most programs don't need that.  Most programs want to run fast.
> Also, by using System scoped threads, you can monitor your LWPs,
> knowing which is running which thread.

SCS gives predictable realtime scheduling response across the system, but
most programs don't need that. Most programs want to run fast, and you'll
usually get the most efficient execution, and the best management of system
resources, by using PCS threads. "Monitoring" your LWPs might be
comforting, but probably provides no useful understanding of the
application performance. You need an analysis tool that understands the
CONTENTION between your execution contexts, not merely the identity of the
execution contexts. Such an analysis tool can understand PCS threads as
well as SCS threads.

>   Anywhere you're creating threads dynamically, you need to know
> how many threads you're creating and ensure you don't create too
> many.  (Easy to mess up!)  By using a P/C model, you create exactly
> the right number of threads (tuned to machine, CPUs, etc.) and don't
> have to think about them.  If you run at lower than max capacity, having
> a few idle threads is of very little concern.

Remember that "concurrency" is much more useful to most applications than
"parallelism", and is harder to tune without detailed knowledge of the
actual workload. When you're doing I/O, your "concurrency width" is often
far greater than your "execution width". It's often useful, for example, to
dynamically increase the number of "servers" in a process to balance a
large workload, because each server might be blocked on one client's
request for a "long time". Dynamic creation isn't necessarily tied to a
simplistic "thread per request" model.

>   Opinions may be worth what you paid for 'em.

No, no. Opinions are hardly ever worth as much as you paid for them, and
usually a good deal less. Information, however, may be worth far more. One
might hope that in the process of airing our worthless opinions, we have
incidentally exposed some information that might help someone! ;-)

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/
==============================================
 Q175: Multi-platform threading api  

From: "Constantine Knizhnik" 
 

I also have developed threads class library, which provides three system
dependent implementations: based on Posix threads, Win32 and portable
cooperative multitasking using setjmp() and longjmp(). This library was used
in OODBMS GOODS for implementing server and client parts. If somebody is
intersted in this library (and in OODBMS itself), it can be found at

http://www.ispras.ru/~knizhnik


Jason Rosenberg wrote in message <35227A82.3C7059C1@erdas.com>...
>Hello,
>
>I am tasked with converting a set of large core C libraries
>to be thread-safe, and to use and implement a multi-platform
>api for synchronization.  We will need solutions for
>Solaris 2.5.1, Digital Unix 4.0B(D?), Irix 6.2-6.5,
>HP-UX 10.20, AIX 4.1.2+, Windows NT 4.0(5.0) and Windows 95(98).
>
>I have built a basic wrapper api which implements a subset
>of pthreads, and have it working under Digital Unix 4.0B,
>Irix 6.2 (using sprocs), and Windows NT/95.  I am in the
>process of getting it going on the other platforms.


==============================================
 Q176: Condition variables on Win32   
 

Hi,

> I don't see that this is justifiable.

Have you ever seen any of the tortuous attempts by bright fellows like
Jeffrey Richter to define relatively simple abstractions like
readers/writer locks using the Win32 synchronization primitives?  It's
not pretty...  In fact, the first several editions of his Advanced
Windows book were full of bugs (in contrast, we got that right in ACE
using CVs in about 20 minutes...).  If someone of his calibre can't
get this stuff right IN PRINT (i.e., after extensive reviews) I don't
have much faith than garden variety Win32 programmers are going to
have a clue...

> It might be harder if you think in terms of the POSIX facilities. I
> wouldn't say that the combination of a semaphore and mutex or
> critsec is hard though, and the inconvenience of having to acquire
> the mutex after waiting on the semaphore is balanced against
> checking for false wakeups and being unable to signal n resources.

Checking for false wakeups is completely trivial.  I'm not sure what
you mean by "signal n resources".  I assume you're referring to the
fact that condition variables can't be used directly in a
WaitForMultipleObjects()-like API.  Though clearly you can broadcast
to n threads.

> I had in mind that it is allowed to notify more than one thread
> (which always seemed odd to me) but I don't have my POSIX spec
> handy.  Just a nit, but it does stress that false wakeups must be
> handled.

I had it this way originally, but others (hi David ;-)) pointed out
that this was confusing, so I'm grudgingly omitting it from this
discussion.  I suppose anyone who really wants to understand POSIX CVs
ought to read more comprehensive sources (e.g., Bil's book) than my
paper!

> I wish.  In fact they store details about the owning thread and
> support recursive acquisition.  I think this was a screwup by the NT
> designers - critical sections are needlessly expensive.  For very
> basic requirements (I believe) you can get a performance gain using
> the InterlockedIncrement et al for spin locks and an auto reset
> event to release waiters.  (Not sure I can prove it at the moment
> though.  If it makes a measurable difference, you have bigger
> problems than the efficiency of mutexes)

BTW, there's been an interesting discussion of this on the
comp.programming.threads newsgroup recently.  You might want to check
this out.

> This occurs in several examples from 3.1 on.  (Most of them? ...).
> Clear copy-paste bug I'd guess.  Better change to
> ReleaseMutex(external_mutex).

Hum, I'm not sure why you say this.  In all these cases the
"pthread_mutex_t" is typedef'd to be a CRITICAL_SECTION.  Am I missing
something here?

> I think this is rather optimistic.  There is no guarantee that any
> of them will release the mutex in a timely fashion. My original
> objection was to a solution that used a semaphore or other count of
> tokens, and that one thread could loop quickly and steal several
> tokens, leaving threads still blocked.

Right, that was the original discussion that triggered this paper.

> >  EnterCriticalSection (external_mutex);
> Another copy-paste bug.  WaitForSingleObject?

Can you please point out where you think these problems are occurring?
As far as I can tell, everything is typedef'd to be a CRITICAL_SECTION
(except for the SignalObjectAndWait() solution).

Take care,

        Doug
==============================================
 Q177: When stack gets destroyed relative to TSD destructors?  

Douglas C. Schmidt wrote:

>         Can someone please let me know if POSIX pthreads specifies
> when a thread stack gets destroyed relative to the time at which the
> thread-specific storage destructors get run?  In particular, if a
> thread-specific destructor accesses a pointer to a location on the
> run-time stack, will this memory still exist or will it be gone by the
> time the time the destructor runs?

Thread-specific data destructors must run in the context of the thread
that created the TSD value being destroyed. (This is clearly and
unambiguously implied by the standard. That is, while the standard
doesn't explicitly require this, an implementation that called
destructors in some other context would present a wide range of severe
restrictions in behavior that are not allowed by the standard.) Thus, the
stack must exist and remain valid at this point.

After a thread has terminated (having completed calling all cleanup
handlers and destructors), "the result of access to local (auto)
variables of the thread is undefined". E.g., at this point, (possibly
before a pthread_join or pthread_detach), the stack may have been
reclaimed.

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/
==============================================
 Q178: Thousands of mutexes?  

Peter Chapin wrote:

> I'm considering a program design that would involve, potentially, a large
> number of mutexes. In particular, there could be thousands of mutexes
> "active" at any one time. Will this cause a problem for my hosting
> operating system or are all the resources associated with a mutex in my
> application's address space? For example, in the case of pthreads, are
> there any resources associated with a mutex other than those in the
> pthread_mutex_t object? Is the answer any different for Win32 using
> CRITICAL_SECTION objects? (I know that there are system and process limits
> on the number of mutexes that can be created under OS/2... last I knew it
> was in the 64K range).

POSIX mutexes are usually user space objects, so the limit is purely based on
your virtual memory quotas. Usually, they're fairly small. Some obscure hardware
may require a mutex to live in special memory areas, in which case there'd be a
system quota -- but that's not relevent on any modern "mainstream" hardware.

On an implementation with 1-1 kernel threads (AIX 4.3, HP-UX 11.0, Win32), there
must be some "kernel component" of a mutex -- but this may be no more than a
generic blocking channel, with the actual synchronization occurring in
user-mode, so there may be no persistent kernel resources involved. Win32
mutexes are pure kernel objects -- critical sections, I believe, are user
objects with a kernel blocking channel (but I don't know whether the kernel
resource is persistent or dynamic).

Similarly, even on a 2-level scheduling implementation (Solaris, Digital UNIX,
and IRIX), a "process shared" mutex (a POSIX option that allows placing a mutex
in shared memory and synchronizing between processes) requires a kernel blocking
channel: but again, the persistent state may live completely in user-space. A
process private (default) mutex, on a 2-level scheduling implementation, is
almost certainly always a pure user-mode object.

Any more detailed answers will require knowing exactly what O/S (and version)
you intend to use.

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/

==============================================
 Q179: Threads and C++  

 
In article <35406CFF.2CD06334@zko.dec.com>,
  Dave Butenhof  wrote:
>
> Federico Fdez Cruz wrote:
>
> >         When I create a thread using pthread_create(...), can I ask to the
new
> > thread to execute a object method instead other "normal" function?
> >         I have tried this, but it seems like the thread doesn't know
anything
> > about the object. If I write code in the method that doesn't use any
> > member of the object, all goes well; but If I try to access a member
> > function or a member in the object, I get a segmentation fault.
> >         I have seen that when the new thread is executing, "this" is NULL
from
> > inside this thread.
>
> Yes, you can do this. But not portably. There's no portable C++ calling
standard.

[Snipped]

> My advice would be to avoid the practice. POSIX 1003.1c-1995 is a C language
> standard, and there are, in general, few portability guarantees when crossing
> language boundaries. The fact that C and C++ appear "similar" is, in some ways,
> worse than crossing between "obviously dissimilar" languages.

In fact, there are significant advantages in doing MT programming in C++.
All the examples in Butenhof D.'s book follow a model of cleaning up (C++
destructor) at the end of block or on error (C++ exception) and can be
written more elegantly in C++ using object (constructor/destructor) and
exceptions. Of course, you will read Butenhof's book for threads (and not
C++).



> Use a static non-member function for pthread_create, and have it call your member
> function. There's no need to bend the rules for a minor convenience, even when
> you can depend on an implementation where the direct call happens to work.
>

You can use the following method in C++ :

Thread.h
--------
class Thread
{
  pthread_t thread_;
public:
  typedef void* (*StartFunction)(void* arg);
  enum DetachedState { joinable, detached };

  Thread(StartFunction sf, void* arg, DetachedState detached_state);
  void join();
  ...
};
// use this to create a thead with a global function entry.
inline Thread Thr_createThread(Thread::StartFunction sf,
                               void* arg,
                               Thread::DetachedState ds)
{
   return Thread(sf, arg, ds);
}

// This is for C++ class member function (non-static)
template 
class ThrAction
{
public:
   typedef void* (T::*MemberFunction)();

   ThrAction(T* object, MemberFunction mf)
     : object_(object), mf_(mf) {}
   void* run() {
     return (object_->*mf_)();
   }
private:
   T* object_;
   MemberFunction mf_;
};

// an implementation class : notice the friend
template 
class ThreadImpl
{
  static void* start_func(void* arg) {
    return ((Action*)arg)->run();
  }
  friend Thread Thr_createThread(Action* action,
                                 Thread::DetachedState ds);
};

// You use this to create a thread using a member function
template
inline Thread Thr_createThread(Action* action,
                               Thread::DetachedState ds)
{
  return Thread(ThreadImpl::start_func, action, ds);
}

Now, you can use a C++ member function:

class MyClass
{
  typedef ThrAction Action;
  Action monitor_;
public:
  void* monitor() {  // this will be called in a new thread
    // ...
  }

  MyClass()
    : monitor_(this, monitor)
  {
    // start a thread that calls monitor() member function.
    Thread thr = Thr_createThread(&monitor_, Thread::detached);
    ...
  }




Thread.cc
---------

struct Detached
{
  pthread_attr_t attr_;
  Detached() { pthread_attr_init(..); }
  ~Detached() { pthread_attr_destroy(..); }
};

static Detached detached_attr;

// depends on the static variable detached_attr; so make it
// out-of-line (in Thread.cc file).
Thread::Thread(StartFunction sf, void* arg, DetachedState ds)
{
  pthread_create(&thread_, sf, arg,
                 ds == detached ? detached_attr.attr_, 0);
}


I wrote it in a hurry. I hope it helps.

- Saroj Mahapatra





In article <6riv2p$ui1@espresso.cafe.net>, Kaz Kylheku  wrote:
++ I second that! I also use the wrapper solution. Indeed, I haven't
++ found anyting in the C++ language definition which suggests that static member
++ functions are like ordinary C functions.

You are correct.  Most C++ compilers DO treat static member functions
like ordinary C functions, so it's usually possible to pass static C++
member functions as arguments to thread creation functions.  However,
some compilers treat them differently, e.g., the OpenEdition MVS C++
compiler doesn't allow static member functions to be used where
ordinary C functions are used, which is a PAIN.

BTW, if you program with ACE
(http://www.cs.wustl.edu/~schmidt/ACE.html) it hides all of this
madness from you so you can write a single piece of source code
that'll work with most C++ compilers.

Take care,

        Doug



I thought all those concerned with developing multi-threaded software
using the STL and C++ might be interested in the topic of STL and thread
safety.  I just bought the July/August 1998 issue of C++ Report and
within there is an article concerning the testing for thread safety of
various popular implementations of STL.  These are the published
results:

 STL implementation     Thread-safe?
 ------------------     ------------
 Hewlett-Packard            NO
 Rogue Wave                 NO
 Microsoft                  NO
 Silicon Graphics           YES
 ObjectSpace                YES

========================  But:   ================

You've missed rather a lot of discussion on this topic in the intervening
months. A few supplemental facts:

1) The definition of ``thread safe'' while not unreasonable is also not
universal. It is the working definition promulgated by SGI and --surprise! --
they meet their own design requirement.

2) Hewlett-Packard provided the original STL years ago, at a time when
Topic A was finding a C++ compiler with adequate support for templates.
Thread safety was hardly a major design consideration.

3) Rogue Wave was quick to point out that the version of their code
actually tested was missing a bug fix that had been released earler.
The corrected code passes the tests in the article.

4) Microsoft has been the unfortunate victim of some messy litigation.
(See web link in footer of this message.) If you apply the fixes from:

http://www.dinkumware.com/vc_fixes.html

then VC++ also passes the tests in the article. Its performance also
improves dramatically.

The C++ Report ain't Consumer Reports. Before you buy on the basis
of an oversimplified table:

a) Make sure your definition of ``thread safety'' agrees with what the
vendor provides.

b) Make sure you get the latest version of the code.

P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com/hot_news.html
-- 





> and I would like to implement the functionality of the jave interface
> runnable. Actually I suppose both of the following questions are
> really c++ questions, but I'm more optimistic to find the competent
> audience for my problems here...
>
> My first question is: Why does it make a difference in passing the
> data Argument to pthread_create, whether the function is a member
> function or not?

Because a member function is not a C function. A member function pointer
is a combination of a pointer to the class, and the index of the function
within the class. To make a member function "look" like a C function,
you must make it a static member function.

> In the following code I had to decrement the data pointer address
> (&arg) by one. This is not necessary, if I define run() outside of any
> class.

This is dangerous!! One another compiler, a different platform, or
the next release of the same compiler, this may or may not work.
It is really a happy accident that it does work in this case.The canonical
form for C++ is to pass the address of a static
member function to pthread_create(), and pass the address
of the object as the argument parameter to pthread_create().
the static member function then calls the non-static member
by casting the void *arg to the class type.

[...]


> The second question is: I've tried to use the constructor of Thread to
> start run() of the derived class as the thread. For this I've
> implemented run() in the base class as pure virtual. But I didn't
> succeed because the thread always tried to run the pure virtual base
> function. Why is this?

Because during the constructor of the base class, the object *is*a base
object. It doesn't know that you have derived something
else from it. It is not possible to call a derived class's virtual
functions during construction or destruction of a base class.

[...]

Here is a minimalist emulation of the Java Runnable and Thread
interface: no error checks, many routines left out, no thread groups
and so on.

-----------------------------------------------------------------------

#include 

// ----------------------------------------------------------------------

class Runnable
{
 public:
  virtual ~Runnable();
  virtual void run() = 0;
};

Runnable::~Runnable()
{
}

// ----------------------------------------------------------------------

class Thread
{
 public:
  Thread();
  Thread(Runnable *r);
  virtual ~Thread();

  void start();
  void stop();
  void join();

  virtual void run();

 private:
  static void *startThread(void *object);
  void runThread();

 private:
  Runnable *target;
  // 0=not started, 1=started, 2=finished
  int state;
  pthread_t thread;
};

Thread::Thread()
  : target(0),
    state(0)
{
}

Thread::Thread(Runnable *r)
  : target(r),
    state(0)
{
}

Thread::~Thread()
{
  if (state == 1)
    pthread_cancel(thread);
  pthread_detach(thread);
}

void Thread::start()
{
  pthread_create(&thread, 0, &Thread::startThread, this);
}

void Thread::stop()
{
}

void Thread::join()
{
  void *value = 0;
  pthread_join(thread, &value);
}

void Thread::run()
{
}

void *Thread::startThread(void *object)
{
  Thread *t = (Thread *) object;
  t->runThread();
  return 0;
}

void Thread::runThread()
{
  state = 1;
  if (target)
    target->run();
  else
    run();
  state = 2;
}

// ----------------------------------------------------------------------

#include 

class Test : public Runnable
{
 public:
  void run();
};

void Test::run()
{
  printf("thread run called\n");
}

int main(int argc, char *argv[])
{
  Thread t(new Test);
  t.start();
  printf("thread started\n");
  t.join();

  return 0;
}

> I've run into a trouble when I found out that when I cancel a thread via
> pthread_cancel() than destructors for local object do not get called.
> Surprising :). But how to deal with this? With a simple  thread code
> it would not be a big problem, but in my case it's fairly complex code,
> quite a few STL classes etc. Has someone dealt with such problem and is
> willing to share his/her soltution with me ? I thought I could 'cancel'
> thread via pthread_kill() and raise an exception within a signal handler
> but it's probably NOT very good idead, is it?;)
> Thank you,

  Unfortunately, not surprising.  C++ has not formally decided what to do with
thread cancellation, so it becomes compiler-specific.  The Sun compiler (for 
example) will run local object destructors upon pthread_exit() (hence 
cancellation also).  Others may not.

  I suppose the best GENERAL C++ solution is:

	a) Don't use stack-allocated objects.
	b) Don't use cancellation.

  Otherwise you can simply insist on a C++ compiler that runs the destructors.



--
Ian Johnston, FX Architecture, UBS, Zurich

==============================================
 Q180: Cheating on mutexes  


Hi all!  Howz things goin?  Just got back from the COOTS conference
where I learned all sorts of valuable lessons ("Don't try to match
Steve Vinoski drink for drink", "Snoring during dull presentations
is not appreciated").  As to this question...

  Pretty much everybody's been largely correct, but a little excessive.

  If we define the objective to be "I have a variable which will under
go a single, atomic state change, can I test it without a mutex?"
then the answer is "yes, if you do things right."  In particular,
if you want to go from FALSE to TRUE, and you don't care if you see
the change synchroniously, then you're OK.

  This is how spin locks work, this is how pthread_testcancel works
(at least on Solaris), and both Dave B & I talk about how to use 
this for pthread_once.

  With spin locks, we test the ownership bit until it becomes "free".
Then we do a trylock on it.  If somebody else gets it first, we go
back to spinning.

  With pthread_testcancel() we test the cancellation flag for our
thread w/o a lock.  If it ever becomes true, we exit.  (The setter
will set it to true under mutex protection, so that upon mutex unlock,
the value will be quickly flushed to main memory.)

  With pthread_once(), we'll insert a test BEFORE calling pthread_once,
testing a variable.  If it's true, then we know pthread_once has executed
to completion and we can skip the test.  If it's false, then we need to
run pthread_once(), which will grab the proper lock, and do the testing
under that lock, just in case someone else was changing it at that instant.

  So...  If you're very, very, careful and you don't mind missing the exact
point of initial change...  you can get away with it safely.

-Bil


> > ...  The real
> > trouble is that if you don't use some kind of synchronisation
> > mechanism, the update may not be seen at other CPUs *at all*.
> ...
> 
> Again donning my newbie hat with the point on top, why not?
> 
> For example, might a a pthreads implementation on a distributed-
> memory architecture not propagate global variables to the other
> CPUs at all, in the absence of something like a mutex?

==============================================
 Q181: Is it possible to share a pthread mutex between two distinct processes?  


>
> ie: some way to attach to one like you can attach to shared memory.
>
> Same question for condition variables.

The answer is (as often happens) both YES and NO. Over time, the balance
will shift strongly towards YES.

The POSIX standard provides an option known commonly as "pshared", which,
if supported on your implementation, allows you to allocate a
pthread_mutex_t (or pthread_cond_t) in shared memory, and initialize it
using an attributes object with a specific attribute value, such that two
processes with access to the shared memory can use the mutex or condition
variable for synchronization.

Because this is an OPTION in the POSIX standard, not all implementations
will provide it, and you cannot safely count on it. However, the Single
UNIX Specification, Version 2 (UNIX 98) requires that this POSIX option be
supported on any validated UNIX 98 implementation.

Implementations that provide the pshared option will define the
preprocessor symbol _POSIX_THREAD_PROCESS_SHARED in the  header
file.

For example,

     pthread_mutexattr_t mutattr;

     pthread_mutexattr_init (&mutattr);
     pthread_mutexattr_setpshared (&mutattr, PTHREAD_PROCESS_SHARED);
     pthread_mutex_init (&mutex, &mutattr);

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/

==============================================
 Q182: How should one implement reader/writer locks on files?  

> How should one implement reader/writer locks on files?
> The locks should work accross threads and processes.

The only way to lock "a file" is to use the fcntl() file locking
functions. Check your man page. HOWEVER, there's a big IF... these locks
are held by the PROCESS, not by the THREAD. You can't use them to
control access between multiple threads within a process.

If you are interested in a mechanism outside the file system, you could
use UNIX98 read/write locks, with the pshared option to make them useful
between processes (when placed in shared memory accessible to all the
processes). However, UNIX98 read/write locks are not currently available
on most UNIX implementations, so you'd have to wait a while. Of course
you'd have to work out a way to communicate your shared memory section
and the address of the read/write lock(s) to all of the processes
interested in synchronizing. Also, while there are ways to make
fcntl() locking mandatory instead of advisory (at least, on most
systems), there's no way to do this with external locking.

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/
==============================================
 Q183: Are there standard reentrant versions of standard nonreentrant functions?  
 

| Certain standard functions found in C programming environments,
| such as gethostbyname, are not reentrant and so are not safe
| for use by multithreaded programs.  There appear to be two
| basic approaches to providing thread-safe versions of these
| functions:
| 
| (1) Reimplement the functions to use thread-local storage.
|     This is the approach that Microsoft has taken.  It's
|     nice because the interface is exactly the same so you
|     don't have to change existing code.

Can you cite documentation that Microsoft has done this consistently? 
(I'd love to be able to rely on it, but haven't been able to pin this
down anywhere.)
 
| (2) Provide alternate reentrant interfaces.  This is the
|     approach taken by (some/most/all?) Unix vendors.  The
|     reentrant version of the function has the same name
|     as the non-reentrant version plus the suffix _r.  For
|     example, the reentrant version of gethostbyname is
|     gethostbyname_r.
| 
| The big problem I'm having with approach (2) is that the
| reentrant versions are not the same across different Unixes.
| For example, the AIX 4.2 and Solaris 2.5 gethostbyname_r
| interfaces are much different (the Solaris interface is
| horrendous, I must say).

FYI, having dealy with this on a couple of Unix systems:  There's the
way Solaris does it, and the way everyone else does it.  While
"everybody else" may not be 100% consistent, the differences are pretty
minor, and often can be worked around with an appropriate typedef.

To be fair to Sun, Solaris probably got there first, and others chose to
do things differently; but that's the end result.  BTW, if you read the
man pages for things like gethostbyname_r, you'll find a notation that
says that the function is "provisional" or something like that, and may
go away in future releases.  There's no change through Solaris 2.6, and
some indication in the release notes that later Solaris versions will
play some games to support both the traditional Solaris API, and the
newer standards - whereever they are drawn from.)

|                          While this is par for the Unix
| course, I'm somewhat surprised that these interfaces are
| not specified by POSIX.  Or are they?  Is there some
| attempt underway to standardize?  Is there some set of
| _r functions that are specified by POSIX, and if so,
| where can I find this list?

*Some* of them were standardized.  Dave Butenhof's "Programming with
POSIX Threads" lists the following:  getlogin_r readdir_r strtok_r
asctime_r ctime_r gmtime_r localtime_r getgrpid_r getgrpnam_r getpwuid_r
getpwnam_r.  Also, a few functions (ctermid is an example) were declared
thread-safe if certain restrictions were followed.

None of the socket-related calls are on this list.  The problem, I
suspect, is that they were not in any base standard:  They're part of
the original BSD socket definition, and that hasn't made it into any
official standard until very recently.  As I recall, the latest Unix
specifications, like the Single Unix Specification (there really aren't
all that *many* of them, but the names change so fast I, for one, can't
keep up), do standardize both the old BSD socket interface, and the "_r"
variants (pretty much as you see them in AIX).

BTW, standardization isn't always much help:  localtime_r may be in the
Posix standard, but Microsoft doesn't provide it.  (Then again,
Microsoft doesn't claim to provide support for the Posix threads API, so
why would you expect it to provide localtime_r....)  You still have to
come up with system-dependent code.
							-- Jerry
==============================================
 Q184: Detecting the number of cpus  
 

bmarker@cco.caltech.edu wrote:
> 
> I have responding to my own posts but I forgot that NT also defines an
> environment variable, NUMBER_OF_PROCESSORS.  Win95/98 may do so as well.
> 
>                 Bradley J. Marker
> 
> In article <6gaa37$2vk@gap.cco.caltech.edu>, bmarker@cco.caltech.edu writes:
> 
> >Win95 only used a single processor the last I looked and there were no plans
> >for SMP for Win98 that I've heard.  I'd personally love SMP on 98 if it
> >didn't
> >cost performance too much.
> >
> >On NT you can get the processor affinity mask and count the number of bits
> >that are on.  Anybody have a better method?
> >
> >sysconf works on IRIX as well as Solaris.  sysconf(_SC_NPROC_CONF) or
> >sysconf(_SC_NPROC_ONLIN) (on Solaris it is NPROCESSORS instead of NPROC).
> >You
> >probably want on-line.  By the way, on an IRIX 6.4 Origin 2000 I am getting
> >sysconf(_SC_THREAD_THREAD_MAX) equal to 64.  Just 64 threads max?  I have
> >multithreaded test programs running with more threads than that (or they
> >seem
> >to be working, anyway).
> >
> >Anybody know how to control the number of processors the threads run on for
> >IRIX?  I'd like both the non-specific run on N processors case and the
> >specifically binding to the Nth processor case.  With Solaris I've been
> >using
> >thr_setconcurrency and processor_bind.
> >
> >Sorry but I don't know for Digital Unix, IBM AIX, or HP-UX.
> >
> >               Bradley J. Marker

In Win32, GetSystemInfo fills in a struct that contains a count of
the number of processors, among other things.

==============================================
 Q185: Drawing to the Screen in more than one Thread (Win32)  

Note: Followup-to: set to comp.os.ms-windows.programmer.win32

[long post removed, see the thread :-) ]

Maybe I'm wrong(*), but AFAIR:

	You can only draw in a window from the thread which own the
window (the one which creates the window). This is this thread which
receives all the messages targeted to the window into its thread
messages list (each thread receive messages for the window it
creates).
	For what I remember, it runs with TextOut() because the second
thread send a message to the first one (which own the window) which
then do the job.
	So if you use a locking mechanism between the two threads for
accessing the window, you may go to a deadlock (thread 2 waiting for
thread 1 painting, and thread 1 waiting for thread 2 releasing the
access).

	Maybe by defining a user message with adequate parameter, and
posting it (the second thread become then immediatly ready to continue
number crunching) into the thread 1 message list, you can achieve a
good update and a minimum thread 2 locking.

A+

Laurent.

==============================================
 Q186: Digital UNIX 4.0 POSIX contention scope  

> I recently found myself at the following website, which describes the
> use of pthreads under Digital Unix 4.0.  It is dated March 1996, so
> I am wondering how up to date it is.
>
> http://www.unix.digital.com/faqs/pub
> ications/base_doc/DOCUMENTATION/HTML/AA-Q2DPC-TKT1_html/thrd.html
>
> It refers to several unimplemented optional funtions from Posix
> 1003.1c 1995, including pthread_setscope.  So I am wondering, then,
> what sort of "scope" do dec pthreads have, are they all system level,
> or all process level, etc.

Digital UNIX 4.0 (through 4.0C) did not support POSIX contention scope.
It was just one of those things that "missed the cut". All POSIX threads
are process contention scope (PCS). Digital UNIX 4.0D supports the scope
attribute. (Since 4.0D has been shipping for some time, it appears that
the web link you've found is not up to date.)

On 4.0D, threads are PCS by default, (as they should be), but you can
create SCS (system contention scope) threads for the rare situations
where they're necessary. (For example, to share realtime resource
directly with hardware, or with OS threads, etc.)


==============================================
 Q187: Dec pthreads under Windows 95/NT?  
> Also, appendix C refers to dec pthreads under Windows 95/NT.  Is that
> a reality?

Depends on what you mean by "reality". Yes, we have DECthreads running
on Win32 "in the lab", and have for some time. In theory, given
sufficient demand and certain management decisions regarding pricing and
distribution mechanism, we could ship it. Those decisions haven't yet
been made. (If you have input on any of this, send me mail; I'd be glad
to forward it to the appropriate person. If you can say whether you're
"just curious" or "want to buy" [and particularly if you can say how
much you'd pay], that information would be useful.)


==============================================
 Q188: DEC current patch requirements  

> It also doesn't describe the current patch requirements, etc., for
> 4.0B.

The Guide to DECthreads is a reference manual, not "release notes", and
is not updated routinely for patch releases. The version you're reading
is clearly for 4.0 through 4.0C, and there's a new version for 4.0D. We
still haven't managed to find time to push through the sticky fibres of
the bureaucracy to get a thread project web page, on which we could post
up-to-date information like current patches and problems.

In general, you should just keep up with the latest patch kit. You can
always keep an eye on the patch FTP directory for your release, under

      ftp://ftp.service.digital.com/public/Digital_UNIX/


==============================================
 Q189: Is there a full online version of 1003.1c on the web somewhere?  
> Is there a full online version of 1003.1c on the web somewhere?

No. The IEEE derives revenue from sale of its standards, and does not
give them away. I understand this policy is "under review". It doesn't
really matter, though, unless you intend to IMPLEMENT the standard.
1003.1c is not a reference manual, and if you want to learn how to use
threads, check out a book that's actually written to be read; for
example, my "Programming with POSIX Threads" (Addison-Wesley) or Bil
Lewis' "Multithreaded Programming with Pthreads" (Prentice Hall) [which,
I see, is so popular that someone has apparently stolen my copy from my
office: well, after all, the spine IS somewhat more colorful than my
book ;-) ].

On the other hand, what IS freely available on the web is the Single
UNIX Specification, Version 2, including "CAE Specification: System
Interfaces and Headers, Issue 5", which is the new UNIX98 brand
specification that includes POSIX threads (plus some extensions). This
document includes much of the text of POSIX 1003.1c, though in slightly
altered form. Check it out at

      http://www.rdg.opengroup.org/onlinepubs/7908799/toc.htm

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/

==============================================
 Q190: Why is there no InterlockedGet?  

> ) My question is renewed, however.  Why is there no InterlockedGet and
> ) InterlockedSet.  It seems under the present analysis, these would be
> ) quite useful and necessary.  Their absence was leading me to speculate
> ) that Intel/Alpha/MS insure that any cache incoherence/lag is not
> ) possible.
> 
> InterlockedExchange and InterlockedCompareExchange give you
> combined Get and Set operations, which are more useful.

Unfortunately, InterlockedCompareExchange is not available
under Windows '95, only NT.  But yes, I agree with you....
==============================================
 Q191: Memory barrier for Solaris  

>I was wondering if anyone knew how to use memory barriers 
>in the Solaris environment. I believe that Dave B.
>posted one for the DEC Alphas.

I assume you use Solaris on a Sparc.

First you should decide whether you are programming for Sparc V8 or Sparc 
V9. Buy the appropriate Architecture manual(s) from Sparc International 
(see: http://www.sparc.com/sparc.new/shop/docs.html )

I have not seen the Sparc Architecture manuals on-line. If someone has, I 
would be grateful for a pointer...

The Sparc chips can be set in different modes regarding the memory model 
(RMO, PSO, TSO). You need to understand the concepts by reading the 
Architecture manual (chapters 6 and J in V8, chapters 8, D and J in V9). 
It is also helpful to know which ordering model Solaris uses for your 
process.

In V8, the "barrier instruction" you are looking for is "stbar". You can 
use it by specifying
        asm(" stbar");
in your C code.

In V9, the architecture manual says:

"The STBAR instruction is deprecated; it is provided only for compatibility 
with previous versions of the architecture. It should not be used in new 
SPARC-V9 software. It is recommended that the MEMBAR instruction be used in 
its place."

The deprecated stbar instruction is equivalent to MEMBAR #StoreStore.

In V9, "memory barriers" are done with the membar instructions. As far as I 
can see, there are 12 different types of the instructions, depending on the 
type of memory barrier you want to have (check the architecture manual).

==============================================
 Q192: pthread_cond_t vs pthread_mutex_t  

Jason Mancini wrote:

> I wrote a small program that loops many times,
> locking and unlocking 3 mutexes.  The results are
> 4.2 million mutex lock-unlocks per second.  Doing the same
> for two threads that wait and signal each other results
> in 26,000 wait-signals per second using conditional
> variables.

Of course, this information is largely useless without knowing what
hardware and software you're using. But nevermind that -- it probably
doesn't matter right now that the numbers are meaningless.

> Any explanations as to why conds are so much slower
> than mutexes?  There are no collisions in any of the
> mutex acquisitions.  Also it seems like the mutex rate
> should be higher that it is.

So, you're trying to compare the performance of UNCONTENDED (that is,
non-blocking) mutex lock/unlock versus condition variable waits. Note
that waiting on a condition variable requires a mutex lock and an
unlock, PLUS the wait on a condition variable. Waking a thread that's
waiting on a condition variable also requires locking and unlocking the
same mutex (in order to reliably set the predicate that must be tested
for a proper condition wait). (If you're not locking in the signalling
thread, then you're doing it wrong and your measurements have no
relevance to a real program.)

Why, exactly, would you expect the performance of the condition variable
protocol to be equivalent to the mutex protocol that consists of a small
part of the condition variable protocol -- and, most importantly, that
excludes the actual BLOCKING part of the condition variable protocol?

As for the mutex rate -- 4.2 million per second means that each
lock/unlock pair takes less than 1/4 of a microsecond. Given the
inherent memory system costs of instructions intended to allow
synchronization on a multiprocessor, you'd need to be running on a
REALLY fast machine for that number to be "bad".

> Is there anything faster available for putting many
> threads to sleep and waking them up many times a
> second?

If your 26,000 per second rate isn't good enough, then the answer is
"probably not". Still, by my count, that's way up in the range of "many
times a second". What exactly are you attempting to accomplish by all
this blocking and unblocking, anyway? If you're doing it as a
consequence of some real work, then what's important is the performance
of the WORK, not the cost of individual operations involved in the work.
(You should really be trying to AVOID blocking, not worrying about
blocking faster, because blocking will always be slower than not
blocking.)

Synchronization is not the goal of multithreaded programming. It's a
necessary evil, that's required to make concurrent programming work.
Synchronization is pure overhead, to be carefully minimized. Every
program will run faster without synchronization... unfortunately, most
concurrent programs won't run CORRECTLY without it.

/---------------------------[ Dave Butenhof ]--------------------------\
| Compaq Computer Corporation                     butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/

==============================================
 Q193: Using DCE threads and java threads together on hpux(10.20)  
>I was wondering if anyone here has had any experience with
>using dce threads and java threads together on hpux(10.20)
>platform.

I'm presuming that you're using a JavaSoft jvm.


DON'T DO THAT!  DON'T EVEN TRY!!!!!


The DCE threads on hpux 10.20 are a user-space threads package.
The JVM uses a (different!) user-space threads package.

Ne'er the two shall meet.

If you have access to an hpux 11.x box, which has kernel threads,
there is a better chance of it working (not great, but better).

It's been quite a while since I looked inside the JavaSoft JVM,
but I seem to recall that the thread API isn't too ugly; you should
probably use those calls to do your C++ threads, but be warned
that you're in some rocky, unexplored territory.  I've even heard
that there be dragons there...

==============================================
 Q194: My program returns enomem on about the 2nd create.  
>   We just upgraded our Alpha from a 250MHz something to a 500+MHz dual
> processor running Digital Unix V4. My program which previously had no
> problem creating hundreds of threads returns enomem on about the 2nd to
> 4th thread create. DEC support advised increasing maxusers from 128 to
> 512 but to no avail. We've got 2Gig of memory and some other sys

The real question is, how much memory does the application use before that
final thread is created? The VM subsystem has an "optimization" for
tracking protected pages that simply doesn't work well with threads. The
thread library always creates a protected page for each stack, to trap
overflows. (You can run without this, by setting the guardsize attribute
to 0... but you shouldn't do that unless you're willing to be money that
your thread won't ever, under any circumstances, overflow the stack;
without the guard page, the results will be catastrophic, unpredictable,
and nearly impossible to debug.)

The problem is that the VM subsystem has a table for dealing with adjacent
pages of differing protection, and it's based on the entire memory size of
the process. If the vm-pagemax parameter is set to 2048, and you have 2048
pages allocated in the process, and you try to protect one of them, the
attempt will fail. If the protection was occurring as part of a stack
creation, pthread_create will return ENOMEM.

While most threaded programs will see this only when they create a lot of
threads (so that the aggregate stack allocation brings the process up over
the vm-vpagemax limit), any program that allocates lots of memory before
thread creation can hit the same limit -- whether the allocation is mmap,
malloc, or just a really big program text or data segment.

So check your vm-vpagemax and make sure it's big enough.

/---------------------------[ Dave Butenhof ]--------------------------\
| Compaq Computer Corporation                     butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/

==============================================
 Q195:  Does pthread_create set the thread ID before the new thread executes?  
 
Wan-Teh Chang wrote:

> The first argument for pthread_create() is the address of a pthread_t
> variable in which pthread_create() will write the new thread's ID before
> it returns.
>
> But it's not clear whether the new thread's ID is written into the
> pthread_t
> variable before the new thread begins to run.

The POSIX standard implies that the ID will not be set before the thread is
scheduled. The actual text is "Upon successful completion, pthread_create
shall store the ID of the created thread [...]." You always need to
remember, in any case, that threads operate asynchronously, and one great
way to hammer that message home is to prevent anyone from counting on
basics like having the create's ID when the thread starts.

(Yeah, that sounds mean, and I guess it is. But way back in the early days
of threading, when nobody know much about using threads, and people yelled
"I don't want to use synchronization, so you need to give me non-preemptive
thread scheduling!", we faced a really big problem in education. I
DELIBERATELY, and, if you like, maliciously, designed the CMA [and
therefore the DCE thread] create routine to store the new thread's id AFTER
the thread was scheduled for execution, specifically so that the thread
will, at least sometimes, find it unset, dragging the reluctant programmer,
kicking and screaming, into the world of asynchronous programming. This was
a purely user-mode scheduler, with a coarse granularity timeslicer, and it
was far too easy to write lazy code that wouldn't work on future systems
with multiple kernel threads and SMP. I couldn't prevent people from
getting away with bad habits that would kill their code later -- but I
could at least make it inconvenient! When I converted to a native POSIX
thread implementation for Digital UNIX 4.0, having battled the education
problem for over half a decade and feeling some reasonable degree of
success, I opted for convenience over forced education -- I set the ID
before scheduling the new thread, and made sure it was documented that
way.)

> I checked the pthread_create() man pages on all major commercial Unix
> implementations, and only the pthread_create(3) man page on Digital Unix
>
> (V4.0D) addresses this issue (and gives an affirmative answer):
>     DECthreads assigns each new thread a thread identifier, which DECthreads
>     writes into the address specified as the pthread_create(3) routine's thread
>     argument.  DECthreads writes the new thread's thread identifier before the
>     new thread executes.
>
> AIX 4.3, HP-UX 11.00, IRIX 6.3, and SunOS 5.6 do not specify the timing
> of the writing of new thread's ID relative to the new thread's execution.
>
> Is this something not specified in the POSIX thread standard?  I don't
> have a copy of the IEEE POSIX thread standard document, so all I can do
> is to read the man pages.  For my application, I need DECthreads'
> semantics that the new thread ID is written before the new thread
> executes.  I guess on other platforms, I will need to have use a mutex to
> block the new thread until the pthread_create() call has returned.

If you really need to code a thread that uses its own ID immediately, you
have a few choices. One, yeah, it can lock a mutex. Just hold a mutex (or,
better, use a condition variable and some predicate) around the
pthread_create call, and treat the thread ID as a shared resource. (Which
it is, although, since it's write-once, and thread create already
guarantees a consistent view of the address space to the created thread,
there's no need for additional synchronization if it's just written before
the new thread is scheduled.) Two, forget about the shared state and just
have the thread call pthread_self(), which will return the exact same ID
that the creator has stored (or will store).

/---------------------------[ Dave Butenhof ]--------------------------\
| Compaq Computer Corporation                     butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/
==============================================
 Q196: thr_suspend and thr_continue in pthread  
 
Niko D. Barli wrote:

> Is there anyway to implement, or to emulate
> Solaris thr_suspend() and thr_continue() in
> pthread ?

Yes, there is. But it's ugly and the result is an inefficient and
complicated wart that reproduces almost all of the severe problems
inherent in asynchronous suspend and resume. If you check the Deja News
archive for this newsgroup, you can probably dig up (much) earlier posts
where I actually told someone where to find suspend and resume code.

> This is the case why I need to use thr_suspend
> and thr_continue.

Think again! You don't need them, and you'll be better off if you don't
use them.

> I have 3 servers running on 3 hosts.
> Each server have 2 threads, each listening to 1 of the other 2 servers.
> Socket information is held in global area, so that every thread can
> access it.
>
> For example, in Server 1 :
>  - thread 1 -> listening to socket a (connection to Server 2)
>  - thread 2 -> listening to socket b (connection to Server 3)
>
> In each thread, I use select to multiplex between socket
> communication and standard input.
>
>   ..........
>
> For example, I ask server 1, to read data from server 2
> (by inputing command from stdin). If my input from stdin
> handled by thread 1, there will be no problem.
> But if thread 2 handle it, thread 2 will send request for
> data to server 2 and waiting. Server 2 will send back data,
> but the data is now listened by BOTH thread 1 and thread 2.
>
> So what I want to do is to suspend thread 1, and let thread 2
> get the data.

You do NOT want to use suspend and resume for this! What you're talking
about is SYNCHRONIZATION between two threads sharing the same resource.
Suspend and resume are NOT synchronization functions, and they won't do
what you want. For example, if you simply depend on asynchronously
suspending one thread "before" it reads from stdin, what if you're late?
(Threads are asynchronous, and, without explicit synchronization, you
cannot know what one is doing at any given time.) What if the thread
you've suspended has already tried to read, and currently has a stdio
mutex locked? Your other thread will simply block when it tries to read,
until the first thread is eventually resumed to complete its read and
unlock the mutex.

Suspend and resume are extremely dangerous and low-level scheduling
functions. You need to know a lot about everything a thread might possibly
be doing before you can safely suspend it -- otherwise you risk damaging
the overall application. (Very likely causing a hang.) If you don't know
every resource a thread might own when you suspend it, or you don't own
every resource YOU might need to do whatever it is you'll do while the
other thread is suspended, then you cannot use suspend and resume. Even if
you do know, and control, all thread, there is always a better and less
dangerous solution than suspend and resume. (Suspend and resume are often
used because they seem convenient, and expedient; and there are even rare
cases where they can be used successfully. But, far more often than not,
you'll simply let your customer find out how badly you've broken your
application instead of finding it yourself.)

/---------------------------[ Dave Butenhof ]--------------------------\
| Compaq Computer Corporation                     butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/
==============================================
 Q197: Are there any opinions on the Netscape Portable Runtime?  


I am working on the Netscape Portable Runtime (NSPR), so
my opinions are obviously biased.   I'd like to provide some info
to help you evaluate this product.

First, NSPR is more than a thread library.  It also includes
functions that are tied to the thread subsystem, most notably
I/O functions.  I/O functions can block the caller, so they must
know which thread library they are dealing with.

The thread API in NSPR is very similar to pthreads.  The
synchronization objects are locks and condition variables.
The NSPR thread API does not have suspend, resume,
and terminate thread functions.  It also does not have the
equivalent of pthread_exit().  NSPR has thread interrupt,
but not thread cancel.  The absence of these functions
in the API is by design.  Also, condition variables are associated
with locks when they are created, and condition notification
must be done while holding the lock, i.e., no "naked" notifies.

The implementation of  NSPR is often merely a layer
on top of the native thread library.  Where there are no native
threads available, we implement our own user-level threads.
NSPR does do a few "value-added" things:
1.  The condition variable notifies are moved outside of the
     critical section where possible.  You must write code
     like this:
           PR_Lock(lock);
           
           PR_NotifyCondVar(queue_nonempty_cv);
           PR_Unlock(lock);
    The actual pthread calls made by NSPR are:
           pthread_mutex_lock();
           
           pthread_mutex_unlock();
           pthread_cond_signal();
    (We use reference count on the condition variables to deal with
    their destruction.)
2. In some two-level thread implementations, a blocking I/O call
    incurs the creation of a kernel schedulable entity (i.e., LWP).  To
minimize
    the number of LWPs created this way, the NSPR I/O functions
    block all the callers on condition variablea, except for one thread.

    A lucky thread is chosen to block in a poll() call on all the
    file descriptors on behalf of the other threads.
3. On NT, NSPR implements a two-level thread scheduler using
    NT fibers and native threads and uses NT's asynchronous I/O,
    while still presenting a blocking I/O API.  This allows you to
    use lots of threads and program in the simpler blocking I/O model.
4. Where it is just too expensive to use the one-thread-per-client
    model but you don't want to give up the simplicity of the blocking
    I/O model, there is work in progress to implement a "multiwait
    receive" API.
    (See http://www.mozilla.org/docs/refList/refNSPR/prmwait.html.)

These are just some random thoughts that came to mind.  Hope it helps.

Wan-Teh
==============================================
 Q198: Multithreaded Perl  
Hello All,

I have just finished to update my Win32 IProc Perl module version
0.15,now i can 
really say that it's a complete module.

Here is all the methods that i have implemented:

    new() (Constructor)
    Create()
    CloseHandle()
    ExitProcess()
    ExitThread()
    FindWindow()
    GetAffinityMask() (WinNT only)
    GetCommandLine()
    GetCurrentHandle()
    GetCurrentId()
    GetExitCode()
    GetExitCodeThread()
    GetThreadStatus()
    GetWorkingSet() (WinNT only)
    GetStatus() (WinNT only)
    GetPriorityClass()
    GetPriorityBoost() (WinNT only)
    GetThreadPriority()
    GetThreadPriorityBoost() (WinNT only)
    Kill()
    LastError()
    Open()
    Resume()
    SetAffinityMask() (WinNT only)
    SetIdealProcessor() (WinNT only)
    SetPriorityBoost() (WinNT only)
    SetPriorityClass()
    SetWorkingSet() (WinNT only)
    SetThreadPriority()
    SetThreadPriorityBoost() (WinNT only)
    SetThrAffinityMask() (WinNT only)
    ShowWindow()
    Sleep() 
    Suspend()
    SwitchToThread() (WinNT only)
    Wait()


With all those 35 methods you will be in a complete control of your 
Threads and Processes.

Add to this my Win32 MemMap that comes with:

 o SysV like functions (shmget,shmread,shmwrite ...)
 o Memory mapped file functions
 ... 

Plus my Win32 ISync module that comes with a complete 
sychronisation mechanisms like: 

 o Mutex
 o Semaphores
 o Events
 o Timers 

 
and the sky will be the limit .

I have included a lot of examples on my modules,i have also updated my
IProc 
Perl documentation,you will find all the doc at:
http://www.generation.net/~cybersky/Perl/iprocess.htm

and all my modules at:
http://www.generation.net/~cybersky/Perl/perlmod.htm


Than you for your time,and have a nice weekend.

Regards
Amine Moulay Ramdane.

"Long life to Perl and Larry Wall!"
==============================================
 Q199: What if a process terminates before mutex_destroy()?  
 
> File locks are released if a process terminates (as the files are closed),

Correct.

> while SYSV-IPC semaphores are persistant across processes,

Unless you specify the SEM_UNDO flag.

> What about (POSIX) mutex's?

There is no cleanup performed on them when a process terminates.
This could affect a mutex (or condition variable) with the process-
shared attribute that is shared between processes.

	Rich Stevens
 

One more point: the "persistence" of an IPC object is different from
what you are asking about, which is whether an IPC object is "cleaned
up" when a process terminates.  For example, using System V semaphores,
they always have kernel persistence (they remain in existence until
explicitly deleted, or until the kernel is rebooted) but they may or
may not be cleaned up automatically upon process termination, depending
on whether the process sets the SEM_UNDO flag.

Realize that automatic cleanup is normally performed by the kernel (as
in the System V semaphore case and for fcntl() record locks) but the
Posix mutual exclusion primitives (mutexes, condition variables, and
semaphores) can be (are normally?) implemented as user libraries, which
makes automatic cleanup much harder.

And, as others have pointed out here, automatic cleanup of a locked
synchronization primitive may not be desireable: if the primitive is
locked while a linked list is being updated, and the updating process
crashes, releasing the locked primitive does not help because the
linked list could be in some intermediate state.  But there are other
scenarios (such as an fcntl() record lock being used by a daemon to
make certain only one copy of the daemon is started) where the automatic
cleanup is desired.

> What about (POSIX) mutex's?  I don't see this documented anywhere.

It's hidden in the Posix specs--sometimes what is important is not
what the Posix spec says, but what it doesn't say.  "UNIX Network
Programming, 2nd Edition, Volume 2: Interprocess Communications"
(available in ~2 weeks) talks about all this.

	Rich Stevens
==============================================
 Q200: If a thread performs an illegal instruction and gets killed by the system...  
> % threads should remain open for the life of the application.  However
> % they could perform an illegal instruction and get killed by the system.
> % I would like for the thread creator to post an error that a thread has
> % died, AND then restart the killed thread.
>
> You don't have to worry about this particular case, since the system will
> kill the entire process for you if this happens. Threads aren't processes.

I've answered many questions, here and in mail, from people who expect that
illegal instructions or segmentation faults will terminate the threads. And
even from people who realize that it will terminate the process, but think
they WANT it to terminate only the thread.

That would be really, really, bad. A quick message to anyone who thinks they
want the process to "recover" from a segv/etc. in some thread: DON'T TRY IT.
At best, you'll just blow up your program later on. At worst, you'll corrupt
permanant external data (such as a database file), and won't detect the error
until much, much later.

Remember that a thread is just an "execution engine". Its only private data
is in the hardware registers of the processor currently executing the thread.
Everything else is a property of the ADDRESS SPACE, not of the thread. A
SIGSEGV means the thread has read incorrect data from the address space. A
SIGILL means the thread has read an illegal instruction from the address
space. Either a pointer (the PC in the case of SIGILL) or DATA in the address
space has been corrupted somehow. This corruption may have occurred within
the execution context of ANY thread that has access to the address space, at
any time during the execution of the program. It does NOT mean that there's
anything wrong with the currently executing thread -- most often, it's an
"innocent victim". The fault lies in the program's address space, and
potentially affects all threads capable of executing in that address space.

There's only one solution: shut down the address space, and all threads
within it, as soon as possible. That's why the default action is to save the
address space and context to a core file and shut down. This is what you want
to happen, and you shouldn't be satisfied with anything less. You can then
analyze the core file to determine what went wrong, and try to fix it.
Meanwhile, you've minimized the damage to any external invarients (files,
etc.)... and, at the very least, you know something went wrong.

In theory, an embedded system might handle a SIGSEGV, determine exactly what
happened, kill any "wayward" thread responsible for the corruption, repair
all data, and continue. Don't even IMAGINE that you can do this on anything
but a truly embedded system. You may be able to detect corruption in your
threads, and in the data under control of your code -- but you link against
libpthread, libc, and probably other libraries. They may have their own
threads, and certainly have LOTS of their own data. You cannot analyze or
reconstruct their data. The process is gone. Forget it and move on with life.

If you need to write a "failsafe" application, fork it from a monitor
process. Do NOT share any memory between them! The parent simply forks a
child, which exec*s the real application. The parent then wait*s for the
child, and if it terminates abnormally, forks a replacement. Either the
parent (before creating the replacement) or the replacement (on startup)
should analyze and repair any files that might have been damaged. And then
you're off and running. Safely.

> % I was going to use the posix call "pthread_join" to wait for thread
> % exits. However using  "pthread_join" does not give the thread id of the
> % thread that has died. Is there a way to do this
> % using another thread command.
>
> Well, you say
>   pthread_join(tid, &status);
>
> and if it returns with a 0 rc, the thread that died was the one with
> id _tid_. Your real problem here is that pthread_join won't return
> until the thread formerly known as tid has gone away, so you can't really
> use it to wait for whatever thread goes away first.

My guess is that John is writing code for Solaris (he wrote the article on
Solaris 2.6), and was planning to use the unfortunate Solaris join-any
"extension".

John, don't do that! It's a really, really, bad idea. Unlike UNIX processes,
there's no parental "line of descent" in threads. It's fine to have a "wait
any" that waits for any CHILD of the calling process, and therefore it seems
obvious to extend this concept to threads. But a thread has no children.
There are just an amorphous set of threads within a process, all equals. You
create threads, say, and a database program you're using creates threads, and
a fast sort library it uses creates more threads. Maybe you're also using a
thread-aware math library that creates more... and perhaps the thread library
has its own internal threads that occasionally come and go. Guess what? Your
"join any" will intercept the termination of the NEXT THREAD IN THE PROCESS
to terminate. It may be yours, or the thread library's, or anyone else's. If
it's someone else's thread, and the creator CARED about the termination of
that thread, you've broken the application. (Yeah, YOU broke it, because
there's nothing the library developer could reasonably be expected to do
about it.)

Generally true statement: Anyone who uses "join any" has a broken process.
The only exception is when you're sure that can not possibly, ever, be any
threads in the process you didn't create. And I don't believe anyone can ever
reasonably be sure of that in a modular programming environment. That is, if
you link against a library you didn't write, you don't know it can't ever use
threads. And if it ever does, you lose.

The POSIX pthread_join() function was nearly eliminated from the standard at
several points during the development of the standard. It's a minimal "helper
function" that does nothing of any particular value. It's utterly trivial to
implement pthread_join() yourself. Combine one part mutex, one part condition
variable, a dash of data; stir, and serve. You want to join with ANY of your
worker threads? No problem. Just add another sprinkle of data to record (if
you care) which thread terminated. You don't even need to add more mutexes or
condition variables, because they can all share one set. (To code a "normal"
pthread_join, you'd usually want each thread to have its own set.) Before
terminating, each thread locks the mutex, stores its termination state
(whatever information you want from it) and (if you want) its ID, then
signals or broadcasts (depending on your desired semantics) the condition
variable. To "join", you just wait (in, of course, a correctly tested
predicated condition wait loop!) for someone to have terminated.

/---------------------------[ Dave Butenhof ]--------------------------\
| Compaq Computer Corporation                     butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/
==============================================
 Q201: How to propagate an exception to the parent thread?  

> Does anyone have or know of  an approach to mixing threads with C++
> exception handling?  Specifically, how to propagate an exception to the
> parent thread.  I can catch any exceptions thrown within a thread by way
> of a try block in the entry function.  The entry being a static class
> member function.  (I know, a "state of sin" wrt to C++ and C function
> pointers, but it works.)  Copying the exception to global (static or
> free store)  memory comes to mind.


It was the original intention of the designers of the C++ language to 
allow one to throw/catch across thread boundaries.  As far as I know 
the recently ratified ISO C++ standard makes no mention of threads 
whatsover.  (The ISO C++ committee considered threads an OS and/or 
implementation, not a language issue.  BTW please don't send me a 
bunch of replies agreeing or disagreeing, I am not on the ISO 
committee, I am just reporting the facts. ;-)  However, I am unaware 
of ANY compiler that implements the ability to throw/catch across 
thread boundaries.  I have also discussed this issue with some 
experienced C++ programmers, and they also are unaware of any compiler
that implements this.  I am told that CORBA allows this if you want to
take that approach.  You many want to repost this in 
comp.lang.c++.moderated.  In general there are some problems with this
approach anyway, simply killing a thread does not cause the C++ 
destructors to be called.  (Again I say generally, because the ISO 
standard makes no mention of threads, there is no portable behaviour 
upon which you may count.)  It is usually better to catch an exception
within the thread that threw it anyway.

Peace

Peter



NB: There is no such thing as a "parent" thread.  All threads are created
equal.  But we know what you mean.  

RogueWave's threads.h++ does a rethrow of exceptions in the user-selected
thread (the assigned "parent").  You may wish to look at that.

-Bil
==============================================
 Q202: Discussion: "Synchronously stopping things" / Cheating on Mutexes  




William LeFebvre wrote:

> In article <354F8C4C.4AF1@LambdaCS.com>, Bil Lewis   wrote:
> >  Practically speaking, the operation of EVERYBODY (?) is that a store
> >buffer flush (or barrier) is imposed upon unlocking, and nothing at all
> >done on locking.
>
> Well, except the guarantee that the lock won't be obtained until
> the flush is finished.

Actually, that's incorrect. There may be no "flush" involved. That's the whole
problem with this line of reasoning. One side changes data and unlocks a mutex;
the other side locks a mutex and reads the data. That's not a discrete event,
it's a protocol; and only the full protocol guarantees visibility and ordering.

I dislike attempts to explain mutexes by talking about "flushes" because while a
flush will satisfy the requirements, it's not a minimal condition. A flush is
expensive and heavy-handed. All that's required for proper implementation of a
POSIX mutex is an Alpha-like (RISC) memory barrier that prevents migration of
reads and writes across a (conceptual) "barrier token". This affects only the
ordering of memory operations FROM THE INVOKING PROCESSOR. With a similar memory
barrier in the correct place in mutex lock, the protocol is complete. But with
only half the protocol you get ordering/visibility on one side, but not on the
other; which means you haven't gotten much.

As implied by the quoted statement above, once you've GOTTEN the mutex, you can
be sure that any data written by the previous holder, while the mutex was
locked, has also made its way to the memory system. The barrier in unlock
ensures that, since the unlocked mutex value can't go out before the previous
data goes out; and the barrier in lock ensures that your reads can't be issued
before your mutex lock completes. But this assurance is not necessarily because
of a "flush", and the fact that someone else unlocked a mutex after writing data
is not enough to ensure that you can see it; much less that you can see it in
the correct order.

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/


Subject: Re: synchronously stopping things

David Holmes wrote:

> saroj@bear.com wrote in article <6injjb$u44$1@nnrp1.dejanews.com>...
> > One place where I try to avoid a mutex (at the risk of being called a
> fool)
> > is in singletons:
> >
> > MyClass* singleton(Mutex* mutex, MyClass** instance)
> > {
> >   if (*instance == 0) {
> >     mutex->lock();
> >     if (*instance == 0)  // must check again
> >       *instance = new MyClass();
> >     mutex->unlock();
> >   }
> >   return *instance;
> > }
>
> This coding idiom is known as the "Double Checked Locking pattern" as
> documented by Doug Schmidt (see his website for a pointer to a paper
> describing the pattern in detail). It is an optimisation which can work but
> which requires an atomicity guarantee about the value being read/written.
>
> The pattern works as follows. The variable being tested must be a latched
> value - it starts out with one value and at some point will take on a
> second value. Once that occurs the value never changes again.
>
> When we test the value the first time we are assuming that we can read the
> value atomically and that it was written atomically. This is the
> fundamental assumption about the pattern. We are not concerned about
> ordering as nothing significant happens if the value is found to be in the
> latched condition, and if its not in the latched condition then acquiring
> the mutex enforces ordering. Also we do not care about visibility or
> staleness.

The last part is critical, and maybe rather subtle. You have to not care about
visibility or latency.

So... the code in question is broken. It's unreliable, and not MP-safe at all.
That is, it's perfectly "MT" [multithread] safe, as long as you're on a
uniprocessor... but move to an aggressive MULTIPROCESSOR, and it's "game
over".

Why? Yeah, I thought maybe you'd ask. ;-)

The problem is that you're generating a POINTER to an object of class MyClass.
You're creating the object, and setting the pointer, under a mutex. But when
you read a non-NULL value of the pointer, you're assuming that you also have
access to the OBJECT to which that pointer refers.

That is not necessarily the case, unless you are using some form of explicit
synchronization protocol between the two threads, because having set the value
under a mutex does not guarantee VISIBILITY or ORDERING for another thread
that doesn't adhere to the synchronization protocol.

Yes, "visibility" might seem not to be an issue here -- either the other
thread sees the non-NULL value of "instance", and uses it, or it sees the
original NULL value, and rechecks under the mutex. But ORDERING is critical,
and it's really a subset of VISIBILITY.

The problem is that the processor that sees a non-NULL "instance" may not yet
see the MyClass data at that address. The result is that, on many modern
SMP systems, you'll read garbage data. If you're lucky, you'll SEGV, but you
might just accept bad data and run with it... into a brick wall.

The more aggressive your memory system is, the more likely this is to occur.
You wouldn't, for example, have any problem running on an Alpha EV4 chip...
but on an EV5 or EV6 SMP system, you'll probably end up with intermittent
failures that will be nearly impossible to debug, because they'll often depend
on nanosecond timing factors that you can't reproduce reliably even in
production code, much less under a debugger. (And if you slip by that
"probably" and miss the races, you can be sure that one of your customers will
run into one eventually... and that's even less fun.)

You can fix this problem very simply without a mutex, but that solution is
machine dependent. For example, using DEC C on a Digital UNIX Alpha system, it
could be as simple as changing your test to:

     #include 
         [...]
     if (*instance == 0) {
         [...]
     } else
         asm("mb");

The "mb" (memory barrier) between the test for the non-NULL pointer, and any
later dereferences of the pointer, ensure that your memory reads occur in the
correct and safe order. But now your code isn't portable. And get it wrong in
one place, and your program is toast. That's what I meant in a previous post
about the risk and cost of such optimizations. Is the cost of locking the
mutex really so high that it's worth sacrificing portability, and opening
yourself up to the whims of those ever-more-creative hardware designers?
Sometimes, yes. Most of the time... no way.

/---------------------------[ Dave Butenhof ]--------------------------\
| Digital Equipment Corporation                   butenhof@zko.dec.com |
| 110 Spit Brook Rd ZKO2-3/Q18       http://members.aol.com/drbutenhof |
| Nashua NH 03062-2698  http://www.awl.com/cseng/titles/0-201-63392-2/ |
\-----------------[ Better Living Through Concurrency ]----------------/



From - Sun May 10 01:31:31 1998
From: Achim Gratz 
Newsgroups: comp.programming.threads
Subject: Re: Mutexes and memory systems (was: synchronously stopping things)


OK, I'm trying to wrap things up a bit, although I guess it will still
become rather long.

Forget caching, memory barriers, store buffers and whatever.  These
are hardware implementation details that are out of your control.  You
don't want to know and most of the time you don't know everything
you'd need to anyway, which is more dangerous than knowing nothing.
Trying to infer the specification from the implementation is what gets
you into trouble.

When you lock mutex A, POSIX gives you a guarantee that all shared
data written by whatever thread under mutex A is visible to your
thread, whatever CPU it might run on, and has been completely written
before any reads to it can occur (this is the ordering part).  When
you unlock the mutex, it is best to assume that the shared data
vanishes in neverland.  It is not guaranteed to be up-to-date or
visible at all by POSIX, nor can you infer any order in which the
writes may be performed or become visible.  It is up to the
implementation to employ the hardware in the appropriate manner.
Efficient employment of the hardware is a quality of implementation
issue that's to be considered after the proof of correctness.

[ I don't have the standard, only Dave's book, but that is the
definition that would allow for the most agressive memory
implementations.  It seems sensible, to me at least, to assume this
definition for utmost portability.  It would be interesting to know if
the exact wording does indeed support my interpretation and whether it
was intended to be that strong.  It would seem that for multiple
concurrent readers you'd need to lock multiple mutexes if you want to
stay within the bounds of the above definition. ]


The problem with this definition and the origin (I think) of the
brouhaha in various (usenet) threads here in this group is that
implementation of multiple reader situations seems overly expensive
since a single mutex allows only a single reader at a time, else you
need a mutex for every reader.  All of the schemes presented so far
that propose to avoid the mutex locking by the readers rely on further
properties of the hardware or pthread library implementation.
Fortunately or unfortunately these properties exist on most if not all
hardware implementations in use today or the library implementors have
taken care to slightly expand the guarantees made by POSIX because you
usually don't tell the customer that he is wrong.


Why do these hacks work?

1) no hardware designer can design a system where it takes an
unbounded time for a write to memory to propagate through the system
as that requires infinite resources

2) noone conciously introduces longer delays than absolutely necessary
for reasons of efficiency and stability

3) there is no implementation, AFAIK, of shared memory that is visible
only under an associated mutex, although capability based
architectures might have them

4) in the absence of 3, keeping a directory of the data changed under
each mutex is likely to be more expensive than making all data visible
and introducing order with respect to any mutex except for those that
are still under lock for writing

5) on a shared memory system that lacks 3 and 4, once the data has
been forced to memory by one processor, it is visible to all
processors and no stray stale copies are in caches if the write
occured under mutex lock; even if not ordered on other processors,
data in memory becomes visible after a bounded time because of 1 and 2

6) the above holds for ccNUMA systems as well, although the time for
propagation can be considerably longer


What does not work?

a) any writes of shared data, without locking, atomic or not, with the
exception of "one-shot-flags" (i.e. any value different from the
initial one that has to be set before the threads are started signals
that some event occured and you never change the value back and you
don't care about the exact value itself)

b) multiple changes to a variable without unlocking/locking between
each change may not become visible at all or may be seen to have
different values in different threads

c) any read of shared data, without locking, where accessing data
after the writer has released the lock would be an error or getting
old data can't be tolerated

d) any read of shared data, without locking, that is not properly
aligned or is larger than the size of memory transactions (there may
be several), tearing may occur without notice

e) porting to a system where strict POSIX semantics are implemented
(e.g. NUMA systems with software coherency)


Summary:

You can't do much safely without those pesky mutexes.  The things you
can do aren't, IMHO, in the critical path (performance-wise) most of
the time.  The potential payback is low and the original problem can
be solved by barriers just fine within POSIX (I think - it's been a
long day).  POSIX1003.1j will even give you barriers that may be more
efficient because the library writers took the time to evaluate the
hardware properties in depth.  That suggests that you could indeed
wring some cycles out of these hacks at the expense of portability and
correctness.  If you do, just make sure you don't give this piece of
code to anybody else.

[ If you think I'm paranoid, perhaps I am.  But I'm sick of commercial
software that isn't even linked properly so that it breaks on new
machines and OS releases and sometimes even OS patches when it would
be a simple matter of controlling the build environment to do it
right.  If you have to support more than one of these beauties you get
a very big headache when you find out that the intersection of system
configurations that work is the null set.  Specifications and
standards exist for a reason.  If you don't like them, get them
changed.  Don't break things gratuitously. ]


Achim Gratz.

==============================================
 Q203: Discussion: Thread creation/switch times on Linux and NT   



I'm so excited about this that I had to restate what I now think
to be the key design differences between NT and Linux 2.0 wrt. task
switching:

1.  The design of the Linux scheduler appears to make the assumption that,
    at any time during "normal" operation, there will only be a small
    number of actually runnable processes.

2.  The Linux scheduler computes which of these runnable processes to run
    via a linear scan of the run queue - looking for the highest priority
    process.

3.  The Linux yield_cpu() function is EXTREMELY prejudicial towards the
    calling program.  If you call yield_cpu() you are not only yielding
    the CPU, but you are also setting your priority to zero (the lowest)
    meaning that you will not run again (because of #2 above) until ALL
    other runnable processes have had a bite at the CPU.

4.  A process, under Linux, steadly has its priority lowered as a function of
    how long it has been scheduled to the CPU.

5.  The Linux scheduler re-sets everyone's priority to a "base"
    priority once all of them have had their priority lowered to zero
    (either through #3 or #4).  This re-set entails another linear
    traversal of the run queue in schedule().


Comments:

a:  #1 is probably a very reasonable assumption.

b:  #2 causes task-switching time, on Linux, to degrade as more runnable
    processes are added.  It was obviously a design decision driven by
    the assumption in #1.

c:  #3 is, to me, a contentious issue.  Should you get penalized for
    voluntarily yielding the CPU - should it put you on the "back of the bus"
    or should it simply lower your priority by one?  After all, most other
    voluntary yields (such as for I/O or to sleep for a time) usually
    raise your priority under other UNIXs (don't know if that's the case
    with Linux - haven't checked).

    In either case, Mingo's code changes this policy.

d:  #4 is standard, textbook, OS stuff.

e:  #5 is another reasonable behavior.  The linear scan is, aga