00:10:34 | ajweiss: | has there been any work on any sort of smart wallet layer for doing stuff like payment channels other than what's happening in bitcoinj and bitcore? ie: some kind of wallet/agent layer for pushing around partially signed transactions and defining/running multiparty protocols like payment channels? |
00:11:31 | gmaxwell: | ajweiss: so this was a goal for andytoshi's wizards wallet, but he hasn't gotten terribly far towards any of those ends. |
00:13:42 | ajweiss: | i've been thinking about it a lot. it seems that getting it right would be complicated, as you'd want it to be something that others could implement in differing environments. |
00:27:33 | gmaxwell: | sipa: most of this isn't news to you, the last few slides you may find interesting wrt some of the things we were thinking about in terms of 'algorithmic transparency' for future bitcoin script (though in this case with performance the goal, rather than avoiding unintended behavior) http://cr.yp.to/talks/2015.04.16/slides-djb-20150416-a4.pdf |
00:35:48 | ajweiss: | this reminds me of fftw a bit |
00:37:24 | gmaxwell: | it's funny, FFTW is actually a long way from optimal for many sizes and uses. The really intelligent part of FFTW is only used by its developers: they have written in ocaml a specialized computer algebra system for creating FFT (and FFT like) algorithims; it's responsible for a couple of novel state of the art factorizations. |
00:37:47 | gmaxwell: | None of that is used at runtime though; they precompile a pile of codelets, and the runtime stuff just knows how to try out a couple of mixtures of the codelets. |
00:38:10 | ajweiss: | yep! genfft |
00:38:18 | jcorgan: | animated slides don't translate well to PDFs :( |
00:39:55 | gmaxwell: | ajweiss: I used their ocaml stuff pretty extensively to help with the design of the transforms for the daala video codec. its sadly a bit inscrutible. :) |
00:40:32 | ajweiss: | ocaml is beautiful, when you're the one writing it |
00:42:15 | gmaxwell: | ajweiss: it's not just special algebras though, e.g. we got massive (like 20-30%) speedups in libsecp256k1 just from hand transcribing (with only minimal creativity) straight-forward straight line code from inner kernels into x86 (and arm) ASM. I don't know what drugs people are on with the compilers are smarter talk. That even wasn't SIMD, my expirence with compilers is even worse with SIMD. |
00:52:44 | Taek: | cool pdf. I (and many of my peers/classmates) was under the impression that compilers were better than programming by hand. |
00:55:50 | gmaxwell: | to some extent it depends on whos doing the programming. |
00:56:59 | gmaxwell: | If you are smart though, you'll never do worse (if the compiler is better you'll steal it's approach!)... it does require being clueful about how things work at a low level. |
00:57:31 | gmaxwell: | e.g. if you mistake every computer as a PDP11, the compiler may well outperform you... it at least has a general idea of how to schedule instructions! |
00:58:36 | gmaxwell: | there are a bunch of tricks I know mostly from reading the compiler output. "Why is this faster than me. Oh because you can do it that way. OKAY." |
00:59:39 | gmaxwell: | this is super helpful, because there are some awesome tricks that the compiler can only apply in _some_ contexts; but you can apply them everywhere they make sense. |
01:14:52 | jcorgan: | i think the tradeoff is mostly automation vs. effort |
01:16:13 | gmaxwell: | Yes, though there is another angle; which is that right now, optimization work also usually makes code harder to review (not always though). |
01:17:01 | gmaxwell: | so perhaps in DJB's imagined world where you write simple code then 'interact' with the compiler to produce an optimized version; perhaps then you could be very confident that the optimized version was correct. |
01:31:52 | gmaxwell: | On the optimization point-- on my novena, libsecp256k1 ecdsa verify: |
01:31:53 | gmaxwell: | ecdsa_verify: min 1927us / avg 1928us / max 1929us |
01:32:29 | gmaxwell: | with the hand transliteration to asm of two functions by wumpus: |
01:32:29 | gmaxwell: | ecdsa_verify: min 809us / avg 810us / max 811us |
01:33:11 | jcorgan: | clearly you need gcc --enable-wumpus |
01:34:45 | Apocalyptic: | that's a quite impressive speedup |
01:35:44 | gmaxwell: | Speedup from the similar code on x86_64 isn't as large, but its still impressive. |
01:36:17 | gmaxwell: | This is vs GCC 4.9 w/ O3. Historically GCC has been better tuned on x86 than arm; but x86 generally _needs_ better tuning. |
01:37:59 | gmaxwell: | There isn't any vast wizardy or algorithmic changes, the functions translated are basically straight lines of multiplies and adds; there is no control flow at all. https://github.com/bitcoin/secp256k1/blob/master/src/field_10x26_impl.h#L438 (the verify bits stuff is testing macros, they do nothing in a normal compile) |
01:45:23 | rusty: | gmaxwell: what is wumpus? Google is failing me here... |
01:45:51 | Apocalyptic: | rusty, a bitcoin core dev |
01:46:10 | gmaxwell: | rusty: lol |
01:46:12 | rusty: | Apocalyptic: ah!! Not some cool optimizer :) |
01:46:21 | gmaxwell: | It does sound like a cool optimizer! |
01:46:23 | rusty: | I mean, (s)he may be a cool optimizer... |
01:46:48 | Apocalyptic: | amusingly this is not the first time i've seen that question here or in -dev |
01:55:50 | gmaxwell: | man, what I would pay for a machine executable version of wumpus. |
01:56:38 | rusty: | "gmaxwell threatens to execute wumpus: Does this Dispute Mark the End of Bitcoin Development?" |
01:56:48 | gmaxwell: | lol |
01:57:15 | gmaxwell: | hm. I am sensing a new funding model for Bitcoin development; write trolly clickbait articles for coindesk ... profit! |
02:00:18 | gmaxwell: | would make for a fun dilbert (or perhaps XKCD beret guy) where the boss is confused and thinks that staff can be "scaled out" using AWS. |
02:00:41 | gmaxwell: | "If we get overloaded we can just spin up another 50 BOB instances!" |
02:02:40 | jcorgan: | http://www.analogsf.com/0310/cookie.shtml |
02:03:14 | gmaxwell: | jcorgan: I assume thats cookie monster? |
02:03:23 | jcorgan: | yep |
02:03:56 | gmaxwell: | Must read. |
02:04:37 | jcorgan: | one of vinge's best |
02:04:50 | gmaxwell: | ah, whole thing isn't there. |
07:35:50 | mm_0: | mm_0 is now known as mm_1 |
08:05:17 | sinisalo.freenode.net: | topic is: This channel is not about short-term Bitcoin development | http://bitcoin.ninja/ | This channel is logged. | For logs and more information, visit http://bitcoin.ninja |
08:05:17 | sinisalo.freenode.net: | Users on #bitcoin-wizards: andy-logbot Mably priidu cbeams HostFat damethos nivah x98gvyn jhogan42 whale xcthulhu hulkhogan42o b_lumenkraft NewLiberty RoboTeddy p15x unlord_ Crowley2k [7] moa GAit justanotheruser Dr-G hashtagg rusty afk11 DougieBot5000 BananaLotus guruvan nessence Starduster Emcy lmacken SDCDev dgenr8 rustyn harrigan cluckj p15 Firescar96 Pan0ram1x sparetire PaulCapestany jeremyrubin dc17523be3 Tjopper Relos arubi_ antgreen waxwing spinza Eliel OneFixt |
08:05:17 | sinisalo.freenode.net: | Users on #bitcoin-wizards: shesek cornus_ammonis airbreather PRab ryanxcharles Cory melvster K1773R kgk LeMiner tjader warren jtimon c0rw1n grandmaster xapp Alanius brand0 poggy ajweiss nuke1989 Zouppen Krellan stonecoldpat phedny aakselrod Guest4827 s1w lechuga__ devrandom EasyAt_ yorick afdudley mm_1 Taek lnovy [d__d] cdecker gielbier jmaurice leakypat Tiraspol platinuum koshii MoALTz isis smooth sneak SubCreative Madars tromp_ throughnothing_ amiller sparetire_ |
08:05:17 | sinisalo.freenode.net: | Users on #bitcoin-wizards: gmaxwell maaku Xzibit17 adams_ forrestv Transisto richardus adlai Fistful_of_coins go1111111 prodatalab morcos sdaftuar andytoshi helo Iriez copumpkin bedeho face Kwelstr nsh cfields_ wumpus mappum jbenet NeatBasisW dasource dardasaba ebfull mkarrer_ luigi1111w binaryatrocity bliljerk101 jonasschnelli merlincorey [ace] eric a5m0 nephyrin null_radix crescendo Sqt mikolalysenko sturles GreenIsMyPepper harrow vonzipper berndj manan19 comboy |
08:05:17 | sinisalo.freenode.net: | Users on #bitcoin-wizards: jaromil catlasshrugged_ Apocalyptic cryptowest_ runeks__ Graet veox indolering Keefe petertodd jcorgan larraboj ryan-c jessepollak gribble tromp mr_burdell d9b4bef9 starsoccer weex SwedFTP Hunger- Luke-Jr yrashk artifexd kumavis otoburb huseby midnightmagic BlueMatt TD-Linux mariorz hguux___ wizkid057 Anduck kyuupichan Logicwax phantomcircuit luigi1111 nanotube yoleaux gavinandresen AdrianG BrainOverfl0w MRL-Relay azariah @ChanServ davout |
08:05:17 | sinisalo.freenode.net: | Users on #bitcoin-wizards: CryptOprah Oizopower epscy nickler gwillen kinlo coryfields_ Muis catcow sl01 null michagogo STRML warptangent pigeons espes__ roasbeef_ dansmith_btc cursive Meeh fluffypony optimator livegnik |
08:07:13 | fluffypony: | btw jcorgan we got a 90% efficiency increase when compiling Monero with --enable-wumpus |
08:19:26 | mm_1: | mm_1 is now known as mm_0 |
09:20:27 | fanquake: | fanquake has left #bitcoin-wizards |
10:58:50 | Mably_: | Mably_ is now known as Mably |
11:27:14 | Eliel: | fluffypony: is that an inside joke or are you serious? :P |
11:57:54 | fluffypony: | Eliel: https://botbot.me/freenode/bitcoin-wizards/2015-04-18/?msg=36911496&page=1 |
12:35:52 | mm_0: | mm_0 is now known as mm_1 |
13:35:42 | Guest4827: | Guest4827 is now known as HM2 |
13:36:06 | HM2: | HM2 is now known as HM |
13:57:23 | mm_1: | mm_1 is now known as mm_0 |
14:22:11 | mm_0: | mm_0 is now known as mm_1 |
14:48:59 | mm_1: | mm_1 is now known as mm_0 |
15:51:07 | delitzer: | delitzer has left #bitcoin-wizards |
16:12:12 | mm_0: | mm_0 is now known as mm_1 |
16:17:48 | fluffypony: | So just so we all know, Ethereum will be 100% safe to use, because it depends on Go code, not on correct maths: https://twitter.com/vitalikbuterin/status/589337931283832832 |
16:17:54 | fluffypony: | .title |
16:17:55 | yoleaux: | Vitalik Buterin auf Twitter: "@fluffyponyza @mperklin The safety of people's funds depends on the go code, not the math notation. And the go code is doing just fine." |
16:21:08 | sturles: | I believe it says "math notation", not maths. |
16:21:43 | fluffypony: | sturles: you can't really separate the two; by definition maths requires precision |
16:22:26 | fluffypony: | "Oh sorry guize, missed landing on Mars because sqrt not sqr, my bad lol" |
16:23:11 | sturles: | Yes, you can. There are several notations for may sub-diciplines in math. |
16:23:11 | fluffypony: | It's basically one of those places where pedantry is a prerequisite:-P |
16:23:24 | sturles: | E.g.: http://en.wikipedia.org/wiki/Notation_for_differentiation |
16:23:51 | sturles: | "In differential calculus, there is no single uniform notation for differentiation. Instead, several different notations for the derivative of a function or variable have been proposed by different mathematicians. The usefulness of each notation varies with the context, and it is sometimes advantageous to use more than one notation in a given context. The most common notations for differentiation |
16:23:57 | sturles: | are listed below." |
16:24:10 | fluffypony: | sturles: sure, but this is the specific case he's referring to - http://imgur.com/MPrtgdy |
16:25:56 | fluffypony: | Tbh his use of "notation" is a red herring, this is more than just notation |
16:26:54 | sturles: | Did anyone find actual errors, or just awkward notation? Of course using your own notation may make it very difficult for trained mathematicians to spot mistakes without cleaning it up first. |
16:27:13 | sturles: | Which is bad. |
16:40:46 | gmaxwell: | sturles: most of that paper just makes no sense. |
16:42:24 | gmaxwell: | asking if there are errors is like asking if you found errors in beat poetry. The reason people are picking on the notation is because they feel it's the paper being intetionally obfscuated and putting on a show of sophication that doesn't actually fit. |
16:49:42 | mm_1: | mm_1 is now known as mm_0 |
16:50:51 | fluffypony: | sturles: the notation makes everything unclear, so it's impossible to determine validity or soundness; how do you forego that and then focus on the "concepts"? |
16:51:22 | fluffypony: | put another way: if I told you that 10 minute blocks are way too long, and 8 second blocks are preferable, you'd undoubtedly ask me to provide evidence of that statement |
16:51:43 | fluffypony: | if my evidence is 3WAFFLE 8 X &@#)@#* = 7 what are you going to say? |
16:52:19 | gmaxwell: | It also fails to cite prior work, so it's unclear whats background and what they're claiming is concepts. The things like fraud proofs are an old idea. |
16:53:28 | gmaxwell: | The descriptions of things use symbology that is introduced nowhere before randomly, so you litterally cannot understand quite a few sentences; beyond "I know this is talking about subject X, but I can't figure out what if anything of substance its saying about it.". |
16:54:31 | jcorgan: | i'll give him a small benefit of the doubt. it seems like something a self-taught person would do, unaware of the typical academic conventions of notation, citing prior work, and extending existing conceptual frameworks. |
16:54:58 | jcorgan: | iow, young and naive. |
16:55:57 | fluffypony: | jcorgan: a self-taught person would tend to explain things a LOT more clearly, because they're had to learn from junk on the Interwebz and books written on a subject (rather than academic text books) |
16:56:03 | gmaxwell: | It's also unaware of the solutions that some of the prior work presents. E.g. the problem with fraud proofs is that if no one can check the data, no one can produce the fraud proof. It waxes on philosophically about this at length but never attempts and solutions or really points out how fatal it is for using fraud proofs for anything related to bandwidth scaling. (Part of the reason we've not |
16:56:09 | gmaxwell: | implemented them even though we proposed them in 2011/2012). |
16:56:53 | gmaxwell: | This is also sad because unawareness of other work means that they weren't aware that the community actually has an interesting and powerful improvment to fraud censorship: |
17:00:10 | gmaxwell: | Problem there is: say a block contains an invalid spend; you're expecting people randomly checking parts to spot it (and have constructed the block so this is possible), once any participant finds fraud they can compactly prove it to everyone transitively connected to them. Hurray. But if no one will give them the fradulent data, they can't sample uniformly, and they just won't see it. Setting th |
17:00:16 | gmaxwell: | ings up so someone can prove their sampling is being blocked seems to be quite hard. |
17:02:28 | gmaxwell: | The improvement we have is this, a party offering a block to the network can be required to code it using a locally decodable rateless error correcting code. So they virtually expand the block up to 'infinite' size, such that if you read approx. $blocksize worth of the the infinite space at random, you can recover the whole block. |
17:03:57 | gmaxwell: | Now, when people fetch, they pick random parts to fetch, and check those parts.. but if the sever transmits more than the total block size in aggregate, the other nodes can colaborate to recover any censored parts. So the total amount transmitted must be limited. |
17:05:28 | gmaxwell: | It's not clear if this is actually useful --- it basically means that any block transmitted by an untrusted peer can only come from sources that have the whole block... which is less useful. But its an interesting area.. and the kind of stuff that anyone who would hope to make progress in this space should know about. |
17:12:32 | fluffypony: | gmaxwell: wouldn't that be useful once blockchain pruning gets added, ie. for fetching pre-pruning block data from those peers that offer it? |
17:14:41 | gmaxwell: | fluffypony: you don't need that for that.. in that case you just fetch it from whomever has it. Thats all above about addressing a specific problem where: (1) Not everone will fetch everything, (2) people will fetch things at random and check and tell others if they find problems (3) the randomness in (2) is essential for security. |
17:16:11 | gmaxwell: | one could use fec to make it harder to lose data completely, but the extra overhead of storing correction data could instead be used to just store more blocks. The correction data approach is slightly more powerful, but I don't see "can't find a block" as being a serious issue. |
17:16:30 | gmaxwell: | (and has to be weighed against the FEC being slow) |
17:16:42 | gmaxwell: | Freenet uses FEC in that manner though. |
17:16:52 | fluffypony: | ah makes sense |
17:23:25 | gmaxwell: | (For those not faimlar with how error correcting codes work, imagine instead of storing block 5 or block 500 you store block 5 xor block 500.. now later your data is helpful to _either_ someone who has 5 and wants 500 but can't find it, OR helpful to someone who has 500 and wants 5-- just as helpful as having the wanted block. But you didn't have to know in advance which of the two would go missi |
17:23:31 | gmaxwell: | ng. If 5 and 500 are both missing though, your data isn't helpful at all. There are more complex schemes that let you achieve coding groups of any N=data, K=redundancy, even efficient ones where K=infinity. |
17:32:14 | Taek: | if you're using fountain codes to request random parts of a block, how do you report that someone is refusing to send a particular piece without opening yourself to the attack were someone reports that every piece of every block can't be requested? |
17:32:47 | Taek: | also does using fountain codes give you any sort of guarantee that the block is smaller than $size? |
17:56:15 | gmaxwell: | Taek: you don't report that, instead everyone selects totally seperate pieces (e.g. cryptographically random 128 bit indexes); the sender cannot emit more than a threshold of total output without allowing recovery; if they try to censor any piece that touches a segment of the block in question, they'll have to block almost everyone (basically everyone minus the overhead of the scheme). |
17:59:10 | gmaxwell: | you can think of it interms that the probablity that they'll have to block a user to prevent possible recovery of the fradulent segment starts off at ~1/segments and tends to 1 as the number of segements served goes to the blocksize+overhead. As far as the size goes, you wouldn't be able to decode anything at all if the size was miststated. |
18:05:03 | Taek: | ok. So if I'm trying to decode and verify some subset of the block, I'm going to request cryptographically random coded pieces of the block until I've got enough data to decode some particular segment? |
18:05:30 | Taek: | and if lots of people are doing this, eventually some of them will be able to recover the fraudulent segment? |
18:05:49 | gmaxwell: | Taek: or the server will have to start rejecting virtually every request. |
18:06:06 | gmaxwell: | (by the point where its given out roughly one copy worth in total) |
18:07:17 | Taek: | that's pretty neat |
18:07:54 | gmaxwell: | this may be more useful for something somewhat more centeralized than a blockchain cryptocurrency; e.g. a opentransactions like private ledger, or a system like certificate transparency; ... things where there is a 'well known server'. |
18:08:08 | gmaxwell: | but it's an interesting tool in the toolbelt. |
18:09:55 | Taek: | why couldn't you apply it to lightweight Bitcoin nodes? |
18:10:09 | kanzure: | was there anything before loom and opentransactions that did similar cryptography things for signed receipts and balances? |
18:11:13 | Taek: | it seems to me that you would need multiple parties trying to assemble and share a full block to prevent the server from selectively excluding people by always refusing their requests |
18:11:21 | gmaxwell: | Taek: you can but the server in the scheme needs to have all the data. |
18:11:42 | gmaxwell: | which makes it somewhat less exciting when you want to imagine multiple servers. |
18:11:58 | Taek: | right you still need full nodes, but it seems like an upgrade from SPV verification |
18:12:16 | gmaxwell: | kanzure: digicash? |
18:13:35 | kanzure: | digicash doesn't count for what i have in mind |
18:13:57 | gmaxwell: | Taek: sure, fraud proofs alone are an upgrade there (Even without the anti-censorship measures). (also, note that the bitcoin whitepaper admits spv nodes being able to detect fraud that way; it just doesn't mention compact fraud proofs; so it's unclear how you could detect fraud that way without it being a huge DOS vector). |
18:14:19 | kanzure: | surely there was an open-source "signed receipts" utility that existed backwhenever |
18:17:34 | gmaxwell: | Taek: Another approach we had to the censorship problem was just, if you really cannot obtain a segment, you compute a costly hashcash proof "I cannot obtain segment X". And nodes have some acceptable overhead they're willing to take for additional load for censorship proofs. e.g. they'll double their bandwidth by requesting an equal number of claimed censored segments to actual segements. Then |
18:17:40 | gmaxwell: | they take the censored claims they've heard and pick that many at random weighed by their amount of hashcash proof. If they also find it censored, they begin working on the hashcash to emit a stronger hashcash proof for censorship. |
18:18:11 | gmaxwell: | kanzure: ask adam, nothing is coming to mind but that may be because I don't know precisely what you're referring to. Does RPOW's tokens fit your criteria? |
18:19:08 | amiller: | kanzure, maybe truledger? |
18:19:31 | amiller: | nvm that is loom |
18:19:39 | kanzure: | rpow tokens might fit, but i just mean more general accounting software with issuances, receipts, transactions... surprisingly, gnucash might come close. |
18:19:58 | amiller: | hrm maybe it's not loom... but came after? |
18:20:07 | kanzure: | http://www.gnucash.org/features.phtml |
18:20:29 | kanzure: | this does not even look multi-user |
18:20:33 | gmaxwell: | Taek: I suspect it may be possible that you can set that up so that a censor is unlikely to be successful (unlikely proportional to the overhead people take) when the attacker has a miniority hashpower; but I haven't chased that idea enough to work out the security argument for it. |
19:01:11 | jcorgan: | i'm not sure whatever happened to the project, but last year there was a group that was going to broadcast new blocks in a data sub-channel of a DVB-T station in Finland. IIRC they were going to use a fountain code to continually send encoded parts of the latest block so that receivers could start decoding at any point in time until they heard enough to reassemble the block locally. |
19:21:40 | pigeons: | truledger can act as a loom client and was inspired by loom, but the actual truledger system uses cryptogrphy instead of loom's "big numbers" |
19:24:01 | pigeons: | truledger server and client agree on signed balances yes |
22:47:26 | mm_0: | mm_0 is now known as mm_1 |
22:47:55 | mm_1: | mm_1 is now known as mm_0 |