00:25:46coinheavy_:coinheavy_ is now known as coinheavy
01:18:55atgreen:jgarzik: re: newlib, you only have to implement a handful of "system calls" in order to get useful code running. For instance _sbrk() will give you malloc & friends. Right now if you try to call malloc the link will fail thanks to the missing _sbrk() implementation. For simplicity sake, you could just have _sbrk() swi back into the interpreter, which will create and grow the heap on demand.
01:59:59jgarzik:atgreen, unfortunately, after having been a kernel developer for 1.5 decades, I've developed a hatred for sbrk(), and am convinced that mmap(MAP_ANONYMOUS) is far superior, and offers the kernel (or sandbox) much more flexibility.
02:00:29jgarzik:atgreen, malloc should be more like glibc's malloc, which uses mmap(MAP_ANONYMOUS)
02:23:04jgarzik:atgreen, What other system calls are required, I wonder...
02:23:28jgarzik:atgreen, a piecemeal, cherry-picking approach to newlib may be better for a sandbox with 2 syscalls ;p
02:57:33atgreen:you can always drop your own malloc in libsandboxrt
03:12:41jgarzik:atgreen, or patch newlib to be sane ;p
03:12:53jgarzik:sbrk() really is deprecated
03:14:19jgarzik:mmap and sbrk are really about allocating regions. I don't want to roll my own malloc(), which adds a layer on top of that.
03:26:44atgreen:newlib has a mechanism to drop in cpu and os specific code to override the defaults
03:27:43atgreen:for instance, if you build newlib for linux, you get an older version of the glibc malloc (which mmaps)
03:28:08atgreen:but the license it that case is lgpl
03:30:48jgarzik:* jgarzik browsers newlib source
03:34:56jgarzik:atgreen, Does not look difficult to modify nano-mallocr.c to work with mmap() instead
03:36:47atgreen:hmm, from arm
03:39:26jgarzik:atgreen, unrelated, a remote gdb stub is beginning to become attractive ;p My sha256 test is either in an infinite loop or taking a Really Long Time :) Time for some good debugging tools.
03:41:02atgreen:ok, I can do that
03:41:22atgreen:I was wondering how long it would take you to get tired of instruction traces
03:42:04atgreen:btw, I was writing them in csv to make them easy to load into a spreadsheet. much easier to view.
03:43:02atgreen:are you OK with including some GPL code that will only be built when configured with --enable-gdb?
03:49:18jgarzik:atgreen, csv, good idea
03:49:26jgarzik:atgreen, no :(
03:50:03atgreen:ok, I have a super tiny stub that should work.
03:51:35atgreen:I'll base it on the one I wrote for my FPGA firmware... https://github.com/atgreen/moxie-cores/blob/master/firmware/bootrom/tinystub.c
03:54:17jgarzik:w00t. my bug, not the sim bug (of course).
03:54:21jgarzik:* jgarzik pushes sha256 test
03:54:26jgarzik:another milestone
03:54:39jgarzik:input: program + random data. output: sha256 hash.
04:56:34jgarzik:odd. make check works, but make distcheck does not. tests/test-suite.log yields proper sandbox output. hmph. :)
04:56:38jgarzik:for another day...
04:58:27jgarzik:atgreen, This is why I hate administering wikis: http://moxielogic.org/wiki/index.php/Special:RecentChanges
04:59:25jgarzik:atgreen, it is worth exploring replacement of my memory map with a Device Tree blob
05:02:37Dr-G2:Dr-G2 is now known as Dr-G
05:15:39jgarzik:* jgarzik kicks $top_srcdir, fixes make distcheck, and tags v0.0.2
05:30:11jgarzik:gmaxwell, it's rudimentary, but moxiebox now runs the simulator in a child process via fork(), too, in addition to having a working sha256 [non-accelerated] test.
07:04:17gmaxwell:maaku: https://github.com/seL4/seL4
07:21:01maaku:gmaxwell: just saw that on HN :)
07:21:23maaku:nice that they have an ARM7 port in there
08:34:47gmaxwell:jgarzik: so one fun thing to do— on a 64 bit host, set up the process memory map so that there are 4gb of unmapped pages above and below the moxie memory, so that no moxie pointer arithemetic could end up pointing to mapped memory in the sandbox process.
08:59:36wallet421:wallet421 is now known as wallet42
10:18:09atgreen:jgarzik: yes, wikis are the worst. :(
10:20:40atgreen:what's the value in fork()ing the simulator?
12:53:05aburan28:anyone checked out this static analysis of the bitcoin code? http://www.viva64.com/en/b/0268/
12:53:32sipa:#bitcoin-dev please; and yes
13:41:53jps_:jps_ is now known as jps
14:47:29tobiaskim:hey, check this out - https://bitcointalk.org/index.php?topic=711966.0;all - Must be something we all can have use for, what do you think?
14:50:04tacotime:tobiaskim, #bitcoin-assets maybe
14:51:08tobiaskim:its a trade-bot/application
14:55:47tacotime:tobiaskim, right, this room is for academic discussion, e.g. papers, maths regarding cryptocurrency
14:59:16tobiaskim:any academic thoughts on trading-algos?
14:59:41tacotime:depnds what you're doing
14:59:51tacotime:eg trying to meet vwap or hft
15:01:52jgarzik:tacotime, ...related projects
15:02:01jgarzik:like agents, oracles
15:02:11jgarzik:speaking of,
15:02:38jgarzik:amiller, it would be fun to dust off that old discussion we had about a more useful DHT, just to publish a proof of concept
15:02:44tacotime:jgarzik, ah, ok
15:03:12tacotime:i guess we do talk a lot of random alts here too, on a technical level
15:05:48randy-waterhouse:jgarzik: i agree re amiller DHT stuff
15:09:20jgarzik:randy-waterhouse, in what regard?
15:10:29randy-waterhouse:let's dust it off
15:11:02randy-waterhouse:publish PoC
15:11:28randy-waterhouse:seems he's been busy on other stuff though ...
15:13:10randy-waterhouse:ha ... just thought that PoC sounds like a new alt-coin mining algo
15:13:40randy-waterhouse:write it in Haskell?
15:15:51jgarzik:randy-waterhouse, amiller and I were talking specifically about building a DHT (not a bitcoin-like D1HT) that has some additional requirement to prove that it is acceptable for you store a new entry in the DHT
15:17:41randy-waterhouse:like what kind of "additional requirement"?
16:04:21zooko`:zooko` is now known as zook
16:04:23zook:zook is now known as zooko
16:27:27jgarzik:randy-waterhouse, something. PoW is an obvious choice, but you can be smarter than that. e.g. depend on a dataset, notably, you could only permit storage of items whose hashes are embedded in the blockchain
16:27:53jgarzik:randy-waterhouse, the minimum requirement is simply a proof, or proofs, that reduce flooding
16:40:36mapppum:are there any p2p protocols that use BTC fidelity bonds for dos+sybil protection?
18:55:40kanzure:petertodd: are you doing an audiot of counterpartyd.git?
18:56:04kanzure:eg https://github.com/CounterpartyXCP/counterpartyd/issues/213
18:56:38kanzure:i had a pile of suggestions that they ignored, maybe i can just streamline them through you (like "unit testing" and "use standard packaging conventions instead of inventing new ones")
18:57:59kanzure:maybe "peter todd audit" means something else though
18:59:43gmaxwell:atgreen: Presumably it gets forked to put it in a seperate memory space, from there, before executing it will drop all privlegeds and get seccomp2. filtered, as an added protection in the case of a sandbox escape. (of more concern if there are some non-trivial native code accelerators added)
19:19:19jgarzik:gmaxwell, indeed, though since you have to load stuff from filesystem before seccomp, a non-fork approach that applies self-limits like seccomp after a certain initialization point seems to provide pretty much the same protections
19:20:11jgarzik:gmaxwell, No objection to forking. I took it out for the moment. Want to think about the threat models a bit.
19:21:30gmaxwell:I doubt its essential, but being able to throughly strip off all connections to the outside world but the bare minimum is interesting. Prior to seccomp this was easiest to do with a fork boundary. Might still be worthwhile. Dunno.
19:24:03jgarzik:gmaxwell, it's certainly true that seccomp is not universal on all platforms
19:24:12jgarzik:gmaxwell, though sandbox itself is a process boundary
19:24:44jgarzik:and ex-seccomp, fork() is not going to limit your syscall ability
19:25:19jgarzik:gmaxwell, I imagine a long-running server would want to fork() for each oracle invocation
19:25:42jgarzik:for a single-run binary, seems like that already happens at a higher level.
19:26:02gmaxwell:jgarzik: how are you doing cyclecount limits right now?
19:26:27jgarzik:gmaxwell, in-simulator instruction count
19:27:17jgarzik:gmaxwell, c.f. "self-limiting" above, I think adding rlimit() calls, chroot() and other self-limiting things in addition to seccomp() should be added.
19:27:43jgarzik:* jgarzik tries to think of seccomp() as icing on the cake, which might not exist on a platform like *bsd.
19:28:59jgarzik:gmaxwell, thus, _ideally_, you would rlimit CPU usage somehow, in addition to the current in-simulator limit. seems difficult to come up with constant or formula for the CPU rlimit.
19:29:34gmaxwell:An alternative execution enviroment might be to run the simulator under qemu-kvm, if run without a kernel it wouldn't have seccomp and such, so you'd want to be able to build without it for that.
19:31:09jgarzik:ultimately I want to target both single-run CLI binary and long-running oracle server use cases. I think the current process limiting tools are "pretty good" But indeed, if you want to get perfect isolation like that, you start adding another layer of simulation.
19:35:10gmaxwell:yea, the kvm route has the advantage that it has ~no performance overhead for this use... though qemu adds a bunch of startup overhead last I checked.
19:39:48gmaxwell:jgarzik: Obviously you also need to indivigually compile each instant of the sandbox with https://github.com/obfuscator-llvm/obfuscator/wiki
19:40:21jgarzik:* jgarzik looks
19:41:09jgarzik:LLVM can do some fun things
19:41:23jgarzik:innovation at the IR level is only starting
19:43:07catia:catia has left #bitcoin-wizards
19:43:12gmaxwell:(I suppose it actually would be reasonable for each oracle to have indivigually obfuscator-llvm'ed their sandbox.
19:43:30gmaxwell:though I dunno if that code will be maintained long term.
19:47:15jgarzik:gmaxwell, hopefully it gets integrated as an optional pass in the main llvm repo
19:47:26jgarzik:that's the normal path for such things
19:50:21jgarzik:gmaxwell, It can also be fun to play around with clone(2). That is your "choose your level of fork" slider, from lightweight thread to heavyweight fork. You can piece together precisely just the bits you need from the parent process.
20:52:57kazcw:kazcw has left #bitcoin-wizards
21:27:26jgarzik:"The new architecture has six instructions: four instructio
21:27:26jgarzik:ns (AESENC, AESENCLAST,
21:27:26jgarzik:AESDEC, and AESDELAST) facilitate high performance AES enc
21:27:26jgarzik:ryption and decryption,
21:27:26jgarzik:and the other two (AESIMC and AESKEYGENASSIST) support the
21:27:27jgarzik:AES key expansion.
21:27:29jgarzik:Together, these instructions provide full hardware support for
21:27:31jgarzik:AES, offering high
21:27:33jgarzik:performance, enhanced security, and a great deal of software u
21:27:37jgarzik:sage flexibility
21:27:45jgarzik:* jgarzik kicks pdf pasting
21:28:25jgarzik:So, key operations inside CPUs are not unheard-of
22:04:14ghtdak:ghtdak has left #bitcoin-wizards
22:10:28Luke-Jr:is there anything like zeromq, but usable out of the box?
22:11:27randy-waterhouse:define "like"
22:11:49Luke-Jr:randy-waterhouse: message bus that takes care of networkign for me? :P
22:12:04randy-waterhouse:high throughput?
22:13:20Luke-Jr:ideally not something that's going to bog down my network
22:13:31randy-waterhouse:there are few out there ... they all have their ups and downs
22:13:36randy-waterhouse:a few
22:15:10sipa:how do you mean 'takes care of networking', how will nodes find eachother?
22:15:30Luke-Jr:sipa: ideally broadcast, but zeromq requires manual node finding :/
22:16:16Luke-Jr:then I need a whole poll loop :p
22:16:16sipa:oh, easy
22:16:27sipa:just put everything in a blockchain *ducks*