| 00:25:46 | coinheavy_: | coinheavy_ is now known as coinheavy |
| 01:18:55 | atgreen: | jgarzik: re: newlib, you only have to implement a handful of "system calls" in order to get useful code running. For instance _sbrk() will give you malloc & friends. Right now if you try to call malloc the link will fail thanks to the missing _sbrk() implementation. For simplicity sake, you could just have _sbrk() swi back into the interpreter, which will create and grow the heap on demand. |
| 01:59:59 | jgarzik: | atgreen, unfortunately, after having been a kernel developer for 1.5 decades, I've developed a hatred for sbrk(), and am convinced that mmap(MAP_ANONYMOUS) is far superior, and offers the kernel (or sandbox) much more flexibility. |
| 02:00:29 | jgarzik: | atgreen, malloc should be more like glibc's malloc, which uses mmap(MAP_ANONYMOUS) |
| 02:23:04 | jgarzik: | atgreen, What other system calls are required, I wonder... |
| 02:23:28 | jgarzik: | atgreen, a piecemeal, cherry-picking approach to newlib may be better for a sandbox with 2 syscalls ;p |
| 02:57:33 | atgreen: | you can always drop your own malloc in libsandboxrt |
| 03:12:41 | jgarzik: | atgreen, or patch newlib to be sane ;p |
| 03:12:53 | jgarzik: | sbrk() really is deprecated |
| 03:14:19 | jgarzik: | mmap and sbrk are really about allocating regions. I don't want to roll my own malloc(), which adds a layer on top of that. |
| 03:26:44 | atgreen: | newlib has a mechanism to drop in cpu and os specific code to override the defaults |
| 03:27:43 | atgreen: | for instance, if you build newlib for linux, you get an older version of the glibc malloc (which mmaps) |
| 03:28:08 | atgreen: | but the license it that case is lgpl |
| 03:28:17 | atgreen: | s/it/in/ |
| 03:30:48 | jgarzik: | * jgarzik browsers newlib source |
| 03:30:52 | jgarzik: | *browses |
| 03:34:56 | jgarzik: | atgreen, Does not look difficult to modify nano-mallocr.c to work with mmap() instead |
| 03:35:38 | atgreen: | ok. |
| 03:36:47 | atgreen: | hmm, from arm |
| 03:39:26 | jgarzik: | atgreen, unrelated, a remote gdb stub is beginning to become attractive ;p My sha256 test is either in an infinite loop or taking a Really Long Time :) Time for some good debugging tools. |
| 03:41:02 | atgreen: | ok, I can do that |
| 03:41:22 | atgreen: | I was wondering how long it would take you to get tired of instruction traces |
| 03:42:04 | atgreen: | btw, I was writing them in csv to make them easy to load into a spreadsheet. much easier to view. |
| 03:43:02 | atgreen: | are you OK with including some GPL code that will only be built when configured with --enable-gdb? |
| 03:49:18 | jgarzik: | atgreen, csv, good idea |
| 03:49:26 | jgarzik: | atgreen, no :( |
| 03:50:03 | atgreen: | ok, I have a super tiny stub that should work. |
| 03:51:35 | atgreen: | I'll base it on the one I wrote for my FPGA firmware... https://github.com/atgreen/moxie-cores/blob/master/firmware/bootrom/tinystub.c |
| 03:54:17 | jgarzik: | w00t. my bug, not the sim bug (of course). |
| 03:54:21 | jgarzik: | * jgarzik pushes sha256 test |
| 03:54:26 | jgarzik: | another milestone |
| 03:54:39 | jgarzik: | input: program + random data. output: sha256 hash. |
| 03:55:33 | atgreen: | great! |
| 04:56:34 | jgarzik: | odd. make check works, but make distcheck does not. tests/test-suite.log yields proper sandbox output. hmph. :) |
| 04:56:38 | jgarzik: | for another day... |
| 04:58:27 | jgarzik: | atgreen, This is why I hate administering wikis: http://moxielogic.org/wiki/index.php/Special:RecentChanges |
| 04:59:25 | jgarzik: | atgreen, it is worth exploring replacement of my memory map with a Device Tree blob |
| 05:02:37 | Dr-G2: | Dr-G2 is now known as Dr-G |
| 05:15:39 | jgarzik: | * jgarzik kicks $top_srcdir, fixes make distcheck, and tags v0.0.2 |
| 05:30:11 | jgarzik: | gmaxwell, it's rudimentary, but moxiebox now runs the simulator in a child process via fork(), too, in addition to having a working sha256 [non-accelerated] test. |
| 07:04:17 | gmaxwell: | maaku: https://github.com/seL4/seL4 |
| 07:21:01 | maaku: | gmaxwell: just saw that on HN :) |
| 07:21:23 | maaku: | nice that they have an ARM7 port in there |
| 08:34:47 | gmaxwell: | jgarzik: so one fun thing to do— on a 64 bit host, set up the process memory map so that there are 4gb of unmapped pages above and below the moxie memory, so that no moxie pointer arithemetic could end up pointing to mapped memory in the sandbox process. |
| 08:59:36 | wallet421: | wallet421 is now known as wallet42 |
| 10:18:09 | atgreen: | jgarzik: yes, wikis are the worst. :( |
| 10:20:40 | atgreen: | what's the value in fork()ing the simulator? |
| 12:53:05 | aburan28: | anyone checked out this static analysis of the bitcoin code? http://www.viva64.com/en/b/0268/ |
| 12:53:32 | sipa: | #bitcoin-dev please; and yes |
| 13:41:53 | jps_: | jps_ is now known as jps |
| 14:47:29 | tobiaskim: | hey, check this out - https://bitcointalk.org/index.php?topic=711966.0;all - Must be something we all can have use for, what do you think? |
| 14:50:04 | tacotime: | tobiaskim, #bitcoin-assets maybe |
| 14:51:08 | tobiaskim: | its a trade-bot/application |
| 14:55:47 | tacotime: | tobiaskim, right, this room is for academic discussion, e.g. papers, maths regarding cryptocurrency |
| 14:58:02 | tobiaskim: | interesting |
| 14:59:16 | tobiaskim: | any academic thoughts on trading-algos? |
| 14:59:41 | tacotime: | depnds what you're doing |
| 14:59:51 | tacotime: | eg trying to meet vwap or hft |
| 15:01:52 | jgarzik: | tacotime, ...related projects |
| 15:02:01 | jgarzik: | like agents, oracles |
| 15:02:11 | jgarzik: | speaking of, |
| 15:02:38 | jgarzik: | amiller, it would be fun to dust off that old discussion we had about a more useful DHT, just to publish a proof of concept |
| 15:02:44 | tacotime: | jgarzik, ah, ok |
| 15:03:12 | tacotime: | i guess we do talk a lot of random alts here too, on a technical level |
| 15:05:48 | randy-waterhouse: | jgarzik: i agree re amiller DHT stuff |
| 15:09:20 | jgarzik: | randy-waterhouse, in what regard? |
| 15:10:29 | randy-waterhouse: | let's dust it off |
| 15:11:02 | randy-waterhouse: | publish PoC |
| 15:11:28 | randy-waterhouse: | seems he's been busy on other stuff though ... |
| 15:11:55 | randy-waterhouse: | https://cs.umd.edu/~amiller/nonoutsourceable.pdf |
| 15:13:10 | randy-waterhouse: | ha ... just thought that PoC sounds like a new alt-coin mining algo |
| 15:13:40 | randy-waterhouse: | write it in Haskell? |
| 15:15:51 | jgarzik: | randy-waterhouse, amiller and I were talking specifically about building a DHT (not a bitcoin-like D1HT) that has some additional requirement to prove that it is acceptable for you store a new entry in the DHT |
| 15:16:52 | randy-waterhouse: | ah |
| 15:17:41 | randy-waterhouse: | like what kind of "additional requirement"? |
| 15:17:52 | randy-waterhouse: | PoW? |
| 16:04:21 | zooko`: | zooko` is now known as zook |
| 16:04:23 | zook: | zook is now known as zooko |
| 16:27:27 | jgarzik: | randy-waterhouse, something. PoW is an obvious choice, but you can be smarter than that. e.g. depend on a dataset, notably, you could only permit storage of items whose hashes are embedded in the blockchain |
| 16:27:53 | jgarzik: | randy-waterhouse, the minimum requirement is simply a proof, or proofs, that reduce flooding |
| 16:40:36 | mapppum: | are there any p2p protocols that use BTC fidelity bonds for dos+sybil protection? |
| 18:55:40 | kanzure: | petertodd: are you doing an audiot of counterpartyd.git? |
| 18:55:43 | kanzure: | *audit |
| 18:56:04 | kanzure: | eg https://github.com/CounterpartyXCP/counterpartyd/issues/213 |
| 18:56:38 | kanzure: | i had a pile of suggestions that they ignored, maybe i can just streamline them through you (like "unit testing" and "use standard packaging conventions instead of inventing new ones") |
| 18:57:59 | kanzure: | maybe "peter todd audit" means something else though |
| 18:59:43 | gmaxwell: | atgreen: Presumably it gets forked to put it in a seperate memory space, from there, before executing it will drop all privlegeds and get seccomp2. filtered, as an added protection in the case of a sandbox escape. (of more concern if there are some non-trivial native code accelerators added) |
| 19:19:19 | jgarzik: | gmaxwell, indeed, though since you have to load stuff from filesystem before seccomp, a non-fork approach that applies self-limits like seccomp after a certain initialization point seems to provide pretty much the same protections |
| 19:20:11 | jgarzik: | gmaxwell, No objection to forking. I took it out for the moment. Want to think about the threat models a bit. |
| 19:21:30 | gmaxwell: | I doubt its essential, but being able to throughly strip off all connections to the outside world but the bare minimum is interesting. Prior to seccomp this was easiest to do with a fork boundary. Might still be worthwhile. Dunno. |
| 19:24:03 | jgarzik: | gmaxwell, it's certainly true that seccomp is not universal on all platforms |
| 19:24:12 | jgarzik: | gmaxwell, though sandbox itself is a process boundary |
| 19:24:44 | jgarzik: | and ex-seccomp, fork() is not going to limit your syscall ability |
| 19:25:19 | jgarzik: | gmaxwell, I imagine a long-running server would want to fork() for each oracle invocation |
| 19:25:42 | jgarzik: | for a single-run binary, seems like that already happens at a higher level. |
| 19:26:02 | gmaxwell: | jgarzik: how are you doing cyclecount limits right now? |
| 19:26:27 | jgarzik: | gmaxwell, in-simulator instruction count |
| 19:27:05 | gmaxwell: | K. |
| 19:27:17 | jgarzik: | gmaxwell, c.f. "self-limiting" above, I think adding rlimit() calls, chroot() and other self-limiting things in addition to seccomp() should be added. |
| 19:27:43 | jgarzik: | * jgarzik tries to think of seccomp() as icing on the cake, which might not exist on a platform like *bsd. |
| 19:28:59 | jgarzik: | gmaxwell, thus, _ideally_, you would rlimit CPU usage somehow, in addition to the current in-simulator limit. seems difficult to come up with constant or formula for the CPU rlimit. |
| 19:29:34 | gmaxwell: | An alternative execution enviroment might be to run the simulator under qemu-kvm, if run without a kernel it wouldn't have seccomp and such, so you'd want to be able to build without it for that. |
| 19:31:09 | jgarzik: | ultimately I want to target both single-run CLI binary and long-running oracle server use cases. I think the current process limiting tools are "pretty good" But indeed, if you want to get perfect isolation like that, you start adding another layer of simulation. |
| 19:35:10 | gmaxwell: | yea, the kvm route has the advantage that it has ~no performance overhead for this use... though qemu adds a bunch of startup overhead last I checked. |
| 19:39:48 | gmaxwell: | jgarzik: Obviously you also need to indivigually compile each instant of the sandbox with https://github.com/obfuscator-llvm/obfuscator/wiki |
| 19:40:21 | jgarzik: | * jgarzik looks |
| 19:41:09 | jgarzik: | LLVM can do some fun things |
| 19:41:23 | jgarzik: | innovation at the IR level is only starting |
| 19:43:07 | catia: | catia has left #bitcoin-wizards |
| 19:43:12 | gmaxwell: | (I suppose it actually would be reasonable for each oracle to have indivigually obfuscator-llvm'ed their sandbox. |
| 19:43:20 | gmaxwell: | ) |
| 19:43:30 | gmaxwell: | though I dunno if that code will be maintained long term. |
| 19:47:15 | jgarzik: | gmaxwell, hopefully it gets integrated as an optional pass in the main llvm repo |
| 19:47:26 | jgarzik: | that's the normal path for such things |
| 19:50:21 | jgarzik: | gmaxwell, It can also be fun to play around with clone(2). That is your "choose your level of fork" slider, from lightweight thread to heavyweight fork. You can piece together precisely just the bits you need from the parent process. |
| 20:52:57 | kazcw: | kazcw has left #bitcoin-wizards |
| 21:27:15 | jgarzik: | https://software.intel.com/sites/default/files/article/165683/aes-wp-2012-09-22-v01.pdf |
| 21:27:26 | jgarzik: | "The new architecture has six instructions: four instructio |
| 21:27:26 | jgarzik: | ns (AESENC, AESENCLAST, |
| 21:27:26 | jgarzik: | AESDEC, and AESDELAST) facilitate high performance AES enc |
| 21:27:26 | jgarzik: | ryption and decryption, |
| 21:27:26 | jgarzik: | and the other two (AESIMC and AESKEYGENASSIST) support the |
| 21:27:27 | jgarzik: | AES key expansion. |
| 21:27:29 | jgarzik: | Together, these instructions provide full hardware support for |
| 21:27:31 | jgarzik: | AES, offering high |
| 21:27:33 | jgarzik: | performance, enhanced security, and a great deal of software u |
| 21:27:37 | jgarzik: | sage flexibility |
| 21:27:45 | jgarzik: | * jgarzik kicks pdf pasting |
| 21:28:25 | jgarzik: | So, key operations inside CPUs are not unheard-of |
| 22:04:14 | ghtdak: | ghtdak has left #bitcoin-wizards |
| 22:10:28 | Luke-Jr: | is there anything like zeromq, but usable out of the box? |
| 22:11:27 | randy-waterhouse: | define "like" |
| 22:11:49 | Luke-Jr: | randy-waterhouse: message bus that takes care of networkign for me? :P |
| 22:12:04 | randy-waterhouse: | high throughput? |
| 22:12:09 | Luke-Jr: | shrug |
| 22:13:20 | Luke-Jr: | ideally not something that's going to bog down my network |
| 22:13:31 | randy-waterhouse: | there are few out there ... they all have their ups and downs |
| 22:13:36 | randy-waterhouse: | a few |
| 22:15:10 | sipa: | how do you mean 'takes care of networking', how will nodes find eachother? |
| 22:15:30 | Luke-Jr: | sipa: ideally broadcast, but zeromq requires manual node finding :/ |
| 22:15:59 | randy-waterhouse: | sockets |
| 22:16:01 | randy-waterhouse: | heh |
| 22:16:16 | Luke-Jr: | then I need a whole poll loop :p |
| 22:16:16 | sipa: | oh, easy |
| 22:16:27 | sipa: | just put everything in a blockchain *ducks* |