[ Home ]
SBCL Internals

The pages on this CLiki-driven site can be edited by anybody at any time. No warranty of any kind can therefore be made; any implied warranties of merchantability or fitness for a particular purpose are expressly disclaimed
[ Home ] [ Recent Changes ] [ About CLiki ] [ Text Formatting ]

RANDOM-MT19937 is, on x86, a VOP that implements the Mersenne Twister random number generator due to Matsumoto and Nishimura. On other platforms, there is an implementation in pure Common Lisp, which is of the order of 10 times slower, and conses. Comments in the SBCL source currently suggest that the x86 VOP may be removed; perhaps, however, it should be implemented in assembler on other platforms?

The algorithm has a home page(!)

WHN: My thinking about removing it was along the lines of replacing it with a new algorithm. I'd be inclined to replace MT with a function which calls a fast free C implementation of Rijndael to encrypt a block, then returns words from that block. But I don't use RNGs at all intensively these days, so I'm not motivated to tweak old code when it seems to work OK.

PVE: I've been working on this random generator for CMUCL and it really it not trivial to check if a generator is any good. I would stick with the MT generator until you find another one that passes all their tests for randomness.

WHN: Yes, I know it's hard to test for subtle correlations in an RNG, and in fact in graduate school I wasted some time debugging a path integral Monte Carlo simulation which turned out to be fine, just a too-good test of the correlations of the RNG I was using. But Rijndael is a crypto algorithm, so if there is any computable way to find correlations in its output, it's broken; and since it's the winner of the AES selection process for new crypto algorithms, if it's broken, many smart people will be very surprised. I'd worry about crypto regulation hassles more than I'd worry about subtle correlations.

All this talk about "calling into C" prompts me to stake out a placeholder page describing call_into_c and how the FFI? works - insofar as anyone knows, of course. My impression is that calling C every so often to set up a block is probably OK, but then you want to return a SAP for the block and pick numbers out of it using plain ordinary Lisp. Maybe I just spend too long staring at it, but I find it hard to believe that all that stack frame munging is a fast thing. - Daniel Barlow

I'd like to make a plea to leave the MT algorithm in, and offer a way to set which algorithm will be used, perhaps by setting the value of a global parameter like *which-random-algorithm*.

(Having an additional dynamic parameter seems prone to surprises. If there were to be an algorithm choice, making it an attribute of the *RANDOM-STATE* would ensure that it is used exactly where desired. -- Kevin Reid)

MT has some very nice properties, and is very fast for x86. This is great for those of us who need quality random numbers quickly for Monte Carlo and soforth. Using an encryption algorithm should work well, but you have questions like, where does the block of data come from, and how do I guarantee repeatability for debugging purposes etc. Another option would be to include the C implementation of the MT algorithm for architectures other than x86.

(The best option might be to figure out why the lisp version is too slow and fix things so it isn't.)

CLiki pages can be edited by anyone at any time. Imagine a fearsomely comprehensive disclaimer of liability. Now fear, comprehensively