14:15:00sinisalo.freenode.net:topic is: This channel is not about short-term Bitcoin development | http://bitcoin.ninja/ | This channel is logged at http://download.wpsoftware.net/bitcoin/wizards/. For questions about the logs talk to andytoshi.
14:15:00sinisalo.freenode.net:Users on #bitcoin-wizards: andytoshi-logbot r0ach vfor jaumeWSN eristisk zooko jps bitesports hearn rdponticelli damethos justusranvier copumpkin mr_burdell lclc spinza samson_ TeoDerich jtimon edgars_ wallet42 drawingthesun UukGoblin Guest24743 atgreen andytoshi Guyver2 llllllllll torsthaldo AaronvanW Guest18523 dansmith_btc edulix justanotheruser dgenr8 nairb mappum super3 midnightmagic px1NbxQzEC jaekwon1 e4xit mortale_ Alanius digitalmagus Aquent coinheavy
14:15:00sinisalo.freenode.net:Users on #bitcoin-wizards: michagogo Krellan Ursium Cory tromp jgarzik pajarillo jchp nikitab Grishnakh maaku melvster LarsLarsen zenojis Sangheili Hunger- go1111111 forrestv bbrittain Emcy tjopper Gnosis tacotime mkarrer Apocalyptic waxwing petertodd crescendo roasbeef Pan0ram1x comboy nanotube helo throughnothing catcow Fistful_of_Coins ranjit_ OneFixt realazthat NikolaiToryzin HM kazcw nickler Mikalv Graet grubles Logicwax Transisto jctb zling_ K1773R phedny so
14:15:00sinisalo.freenode.net:Users on #bitcoin-wizards: DoctorBTC @gmaxwell jaromil wizkid057 nkuttler kinlo quackgyver mmozeiko Eliel_ lechuga_ espes__ ryan-c pi07r kanzure epscy Muis amiller artifexd CryptOprah a5m0 EasyAt Ken` @ChanServ gribble mhanne at0mat SomeoneWeird tromp__ jbenet wumpus sl01 lianj danneu burcin phantomcircuit danielpbarron aynstein_ BlueMatt [\\\] harrow otoburb Anduck zibbo [Derek] Keefe warren asoltys LaptopZZ gwillen rs0 weex altoz poggy Guest79174 sipa optimator_
14:15:00sinisalo.freenode.net:Users on #bitcoin-wizards: iddo quickcoin gigavps jcorgan Kretchfoop jj88 kiddouk abc56889_ pigeons
14:19:08atgreen:jgarzik: you'll want that in a linker script. we can do that easily. I've been preparing some small patches for upstream to add a moxiebox- configuration to the GNU toolchain, so moxiebox-gcc will build with the right options (little endian, -lsandboxrt, etc), and we can easily manipulate the linker script to add special symbols at fixed addresses.
14:19:39atgreen:use the lame hack for now, and I can send you toolchain patches shortly with a better solution.
14:20:36atgreen:(there's lots of precedent for this.. see xbox-{gcc,ld,etc}, ps2-{gcc,ld,etc}, and on and on)
14:25:39atgreen:jgarzik: here's how I do it for some moxie firmware where I have memory mapped device ports on the FPGA. See how the .ports section is fixed at 0xF0000000... https://github.com/atgreen/moxie-cores/blob/master/soc/marin/moxie-marin.ld
14:26:31atgreen:and then here I place the symbols in the .ports section: https://github.com/atgreen/moxie-cores/blob/master/firmware/bootrom/tinystart.S
14:26:45atgreen:or you can define the symbols in the linker script.
15:26:17jgarzik:atgreen, ACK
16:04:07jgarzik:atgreen, I cannot do that in the asm? drat. :)
16:04:39jgarzik:atgreen, it seems a custom linker script cannot be avoided
16:18:49mortale_:mortale_ is now known as mortale
17:23:28justanotheruser:Having a UTXO dependent PoW makes it easy for a hasher to automatically stop mining (or pool hop) if their software realizes their hash power is being used to harm the network. As far as I can see, this drastically reduces the power of mining pools and increases the networks security and even more so, SPV security.
17:24:07justanotheruser:However, isn't there an economy of scale involved with this?
17:25:07justanotheruser:You would be running a fast computer to get UTXO info, that would be fed to all your ASICs. The more ASICs you own, the lower percentage of your costs this UTXO querying machine would be.
17:26:17justanotheruser:Perhaps UTXO querying needs to become gradually faster until it is optimal to integrate it with ASICs? (ie. every hash is dependent on a UTXO query)
17:27:24justanotheruser:This does create another ASIC hurdle from what I can see, which is bad, but it decreases a pools power, which is good.
17:41:50Luke-Jr:justanotheruser: it doesn't decrease pool power any more than just normal decentralised mining..
17:43:01justanotheruser:Luke-Jr: but decentralized mining isn't done by most of the network
17:43:47Luke-Jr:justanotheruser: it could be
17:44:11Luke-Jr:justanotheruser: also, a UTXO-dependent PoW breaks validation
17:44:18Luke-Jr:now nodes need the UTXO to validate blocks
17:44:22Luke-Jr:= no SPV
17:44:36justanotheruser:indeed it could, but since people are maximizing profitability, they will probably not burn their man hours and use their hardware.
17:45:30justanotheruser:at least, that is why I think people currently aren't running full nodes + p2pool
17:46:43justanotheruser:Luke-Jr: how does it break SPV?
17:47:30Luke-Jr:justanotheruser: SPV nodes need to validate block PoW
17:47:35Luke-Jr:but they don't have UTXO set
17:47:42justanotheruser:The UTXO proof could be in the coinbase.
17:47:50justanotheruser:I believe that would still have SPV security
17:48:55justanotheruser:Depending on how many UTXO in the UTXO set are validated, it wouldn't make SPV *that* much more bandwidth-expensive
17:53:10kazcw:how can you prove to an SPV node a utxo exists? showing when it was created is easy, but how would you prove it hasn't been spent since?
17:54:31justanotheruser:kazcw: how can you prove to an SPV node a transaction that they're redeeming is valid?
17:54:37justanotheruser:That is SPV security
17:54:54Luke-Jr:justanotheruser: by proving it's in a block
17:55:23Luke-Jr:SPV nodes trust the miners, but that only works because they can verify the PoW
17:55:50justanotheruser:I see what you're saying now
17:56:04justanotheruser:miners can easily generate blocks because they can make false UTXO
17:56:23justanotheruser:or at least thats what I think you're saying
17:56:48gmaxwell:Fraud proofs resolve that.
18:03:37kazcw:how would a proof that a certain utxo has never existed work? don't you need all tx outputs since genesis to know that?
18:04:39andytoshi:kazcw: if you have a hash-ordered utxo trie then you can do a proof of non-existence (demonstrate the utxos immediately before and after the missing one, and that they're neighbors)
18:05:19andytoshi:fwiw this makes insertion proofs hard to construct, which is probably undesirable since then SPV nodes won't have enough data to produce their own txes
18:05:53andytoshi:otoh proof construction could be done by anyone, perhaps for a fee, and there is no risk in outsourcing the proof construction
18:06:08kazcw:andytoshi: back to the earlier problem, how do you demonstrate any utxos to an spv node?
18:06:59andytoshi:the SPV node would trust that the merkle root in the coinbase is correct. (maybe it could verify correct updates for the past few hundred blocks, if enough info was available in transactions. this would take a lot of space, probably not a feasible idea)...
18:07:17andytoshi:...given this merkle root, you could give a proof of presence or non-presence to the SPV node
18:08:01kazcw:andytoshi: but the PoW is dependent on utxo lookups, how can you trust utxo lookups based on the merkle root in the block
18:08:43andytoshi:the miner would provide the full path of the utxo lookup that give the winning PoW
18:09:45andytoshi:so, SPV nodes would not be in a position to validate this kind of thing because they're trusting the merkle root in the block
18:09:53andytoshi:but they could still verify the hash target, same as they do today
18:10:28andytoshi:hmm, i guess this makes it a lot easier to fool SPV nodes since the main chain difficulty will be lower (since everyone is doing slow utxo lookups)
18:12:19jgarzik:atgreen, exported as a libsandboxrt.a global variable
18:12:42jgarzik:atgreen, I think that accomplishes my goals, and is a relatively clean way to avoid a custom linker script
18:13:55andytoshi:kazcw: if every block header contained (a) a merkle root for the whole UTXOset and (b) a merkle root for just the new UTXOs in this block
18:14:12andytoshi:then you could create proof-of-existence which can be validated by a node holding only the headers
18:16:19gmaxwell:andytoshi: you'd want to just have update proofs and the new root. POW would be verified relative to the prior block.
18:16:46gmaxwell:You don't have to give a full node update proofs, since it already can generate them itself.
18:17:58gmaxwell:perhaps tricker is avoiding the bandwidth of the update proof in the common case that the update is faithful.
18:18:11andytoshi:yeah, that's my thought, the bandwidth load would be immense
18:18:18andytoshi:the proofs would be bigger than the original txes in many cases :P
18:22:38andytoshi:wizards-wallet is a full node which checks for legal UTXOset updates (so full validation except assuming all scripts accept), with current code it can sync up to block 278000 in 20 minutes.. though iirc it'll be another hour or so for the rest :)
18:23:01andytoshi:and this is on an i7. so not the kind of thing that a phone could do
18:32:50andytoshi:i lied. full initial sync just finished in 30min 40sec :D
20:24:15jgarzik:atgreen, I want to use two special registers
20:24:23jgarzik:atgreen, any allocation policy, or just pick two?
20:25:18jgarzik:atgreen, this is for a (pointer, length) pair returned from simulator immediately prior to _exit()
20:27:43jgarzik:atgreen, ssr(reg1, pointer) + ssr(reg2, length) + _exit()
20:51:14jgarzik:struct moxie_memory_map_ent {
20:51:14jgarzik:void *addr;
20:51:14jgarzik:size_t length;
20:51:14jgarzik:char tags[32 - 4 - 4];
20:51:15jgarzik:extern struct moxie_memory_map_ent *moxie_memmap;
20:51:17jgarzik:extern void setreturn(void *addr, size_t length);
20:51:21jgarzik:extern void _exit(int status);
20:51:55gmaxwell:Can someone suggest for me which of the dozen prior bitcoin blocksize threads were your favorites, so I can link to them from the top of https://bitcointalk.org/index.php?topic=709970.msg8025200#msg8025200
20:52:13jgarzik:atgreen, gmaxwell: There's the I/O API. A list of memory ranges accessible to the process upon input, and call setreturn() to pass output data immediately prior to exit()
20:52:39jgarzik:under the hood, setreturn() sets two processor registers to (pointer, length)
20:52:50atgreen:jgarzik: I have a document somewhere.. hang on...
20:53:09jgarzik:atgreen, I picked special registers 6 & 7
20:53:44jgarzik:atgreen, found http://moxielogic.org/wiki/index.php/Architecture
20:54:00atgreen:yes, that's the one
20:54:58jgarzik:atgreen, the sim throws an exception if you (1) set SSR 6 to an invalid pointer [inaccessible memory range], or (2) if SSR 7 is set prior to SSR 6 being set
20:55:14jgarzik:just pushed that out
20:55:25jgarzik:we now have a fully functional (if minimal) sandbox!
20:55:45atgreen:I have a nice moxiebox-gcc toolchain as well.
20:56:18atgreen:I just need to fix two things.. where is the stack, and how big is it? and where is this special address you want to export?
20:56:25atgreen:symbol, I mea
20:56:59atgreen:* atgreen looks it up
20:57:02gmaxwell:andytoshi: thats a lot faster than bitcoin-core. I suppose thats great news.
21:00:27jgarzik:atgreen, gcc default linker script puts the stack at 0x400000
21:00:43jgarzik:atgreen, for the "mapdesc" magic address, I picked a constant slightly less than that.
21:05:09jgarzik:atgreen, https://github.com/jgarzik/moxiebox/blob/master/sandbox.cc#L16
21:05:55jgarzik:atgreen, special addr https://github.com/jgarzik/moxiebox/blob/master/runtime/sandboxrt.h#L14
21:08:04andytoshi:gmaxwell: well, i'm avoiding all EC operations
21:08:46atgreen:jgarzik: instead of a fixed address, why don't you just shove mapdesc in a special register at startup and fish it out at runtime?
21:10:05jgarzik:atgreen, I think that is wise
21:10:09atgreen:I think it makes sense to decouple as much as possible the sandbox program from the actual programs
21:10:42jgarzik:I'll re-use SR #6
21:11:06atgreen:to be honest, same could be said for %sp
21:11:42jgarzik:atgreen, agree. And in general, the less fixed addresses, the better.
21:14:11gmaxwell:andytoshi: sure, but bitcoin core also does to until 279000.
21:18:49andytoshi:oh, cool. the algorithmic magic is in https://github.com/rust-lang/rust/pull/15720 ... i haven't time to really look into it, the #rust folks just told me to switch to their hashmap from my patricia tree
21:19:35andytoshi:which saved some mem but no speed ... but then i realized i could store the UTXOs in the hashmap by value rather than allocating them (couldn't do that in the tree because it made the node size much larger and trees are mostly empty nodes)
21:19:59andytoshi:so i removed those allocations, which got me all this time savings apparently :)
21:21:28gmaxwell:andytoshi: how are you getting the blocks? off a file on disk?
21:21:40andytoshi:requesting them from bitcoind over the LAN
21:21:43andytoshi:over localhost
21:21:53andytoshi:500 at a time
21:22:09gmaxwell:pretty awesome. (I ask because you're fast enough that sha256 for that much data is a non-negligible amount of time)
21:22:44andytoshi:yeah, there is lots of hashing. it is possible that rust-crypto is faster than openssl for SHA256..
21:24:40gmaxwell:sipa was benchmarking the openssl sha256 a bit ago, it's no slouch.
21:26:33andytoshi:i'm bisecting rustc right now, but i'll time bitcoind -reindex when i have an unloaded system
21:29:55andytoshi:i guess this should be in -dev, i'll move there
21:29:55jgarzik:/* load sim memory descriptor base address */
21:29:56jgarzik:gsr $r0, 6
21:29:56jgarzik:st.l $r0, moxie_memmap
21:30:03jgarzik:in crt0.S
21:30:33jgarzik:* jgarzik is still learning the ISA, FYI
21:32:31atgreen:sta.l moxie_memmap, $r0
21:32:49atgreen:"sta" = "store at absolute address"
21:33:57jgarzik:atgreen, works!
21:34:14jgarzik:That was mapdesc. Now for stack.
21:34:41atgreen:gsr $sp, 7
21:34:45atgreen:or similar
21:37:34jgarzik:atgreen, yep
21:38:50atgreen:jgarzik: fwiw, rather than create globals, I would start defining a system API, with moxiebox_get_memmap(), which caches the sr6 in a static local. you may discover the need for other special functions, and definiting an API is a bit nicer than magic globals.
21:42:33jgarzik:atgreen, moxie_memmap is a global variable inside the runtime. Special functions and inputs are passed via that array.
21:43:09jgarzik:atgreen, at startup, you'll walk moxie_memmap[] to detect input parameters, data, and additional code (shared libs)
21:44:21jgarzik:atgreen, the general philosophy being you want the Moxie process to start execution with as much data mapped-and-ready-to-go as possible
21:44:45atgreen:ah, ok - prior to calling main()?
21:44:53jgarzik:atgreen, yes.
21:45:02jgarzik:atgreen, well, qualified yes
21:45:48jgarzik:atgreen, you can see how test1 does a walk: https://github.com/jgarzik/moxiebox/blob/master/runtime/test1.c#L26
21:45:53jgarzik:that's post-main()
21:46:13jgarzik:but the memory map etc. is set up prior to simulator execution start
21:46:24jgarzik:think of the memory map like kernel page tables
21:46:31jgarzik:(+ pages)
21:47:45jgarzik:* jgarzik kicks himself. stacks grow down, dummy.
21:52:32jgarzik:atgreen, much cleaner. pushed out those changes to github.
21:57:33atgreen:nice work! kids just got home... on duty now :)
22:25:57jgarzik:Yay. Milestone. We can go from command line-provided code+data input to Moxie execution to file|stdout output.
22:26:34jgarzik:Updated https://github.com/jgarzik/moxiebox/blob/master/execution-env.md but my brain is pretty cooked after all this staring at assembly and objdump.
22:27:38jgarzik:Basic execution environment milestone reached. A proper testsuite can be built. Next step is "make check" Then on to crypto-specific accelerations.
22:29:18jgarzik:atgreen, ditto. nightly kid circus here was postponed an hour for additional coding, but it's coming like a freight train.
22:42:40gmaxwell:heh. collided with me in fixing the segfault on no arguments. :P
22:43:33jgarzik:gmaxwell, sorry. I'm done coding (cc atgreen), so PR away
22:44:04gmaxwell:it's fine, less code that I might have gotten wrong. :)
22:56:04jgarzik:v0.0.1 tagged in git. GPG signed by jgarzik@pobox.com (not jgarzik@bitpay.com or jgarzik@exmulti.com, which are also PGP identities I use)
23:28:34davispuhh:davispuhh is now known as davispuh
23:29:04o3u:o3u is now known as Fistful_of_coins
23:29:29jmlubin_:jmlubin_ is now known as jmlubin
23:29:34kinlo_:kinlo_ is now known as kinlo
23:52:56EasyAt_:EasyAt_ is now known as EasyAt