--- Log opened Tue Jan 14 00:00:09 2014
00:22 < phantomcircuit> gmaxwell, just take your entire hardfork list and implemetn them
01:12 < justanotheruser1> Is there an alternative to pandora that has a GP or BSD license?
01:12 < justanotheruser1> * Pinocchio
04:12 < adam3us> andytoshi: numerical iterative analysis? perfect one for you: selfish attack?
04:51 < adam3us> is there a EdDSA mailing list? or should i just email DJB?  want to figure out this highbit/constant time 'hack' limitation on composability, thats kind of broken.
07:18 < andytoshi> adam3us: the 'hack' is pretty simple, the second answer here explains it: http://crypto.stackexchange.com/questions/11810/when-using-curve25519-why-does-the-private-key-always-have-a-fixed-bit-at-2254/11818
07:19 < andytoshi> and it's also easy to avoid, even if you want to run in constant time -- but then we'd have a nonstandard implementation
07:21 < andytoshi> it might be worthwhile to bug him and ask what he was thinking, because it really is ugly and you're a name he ought to recognize..
08:28 < adam3us> andytoshi: hmm so neves is saying it doesnt matter, if the code does the defense as the reference does, it is still constant time.  what about the other bits?  a[255]=0,a[254]=1, a[2]=a[1]=a[0]=0
08:28 < adam3us> adam3us: none of those will necessarily be the case after public HD derivation
08:29 < adam3us> adam3us: andytoshi: a[255]=0 maybe ok, as |n| < 255 (the order) i presume from the h=8 (cofactor)
08:36 < andytoshi> adam3us: a[255] it appears is always zero, yeah, we'd be fine there (though why are we "using 255-bit strings" with only 254 actual bits??)
08:36 < andytoshi> and as you say (and gmaxwell has been whining for weeks), bit 254 might not be set after pubic HD derivation, or multisig with additive signatures
08:37 < andytoshi> this seems weird, i dunno why the reference implementation didn't just hardcode the 254 in there
08:38 < andytoshi> maybe add a big comment saying not to change that if you are reimplementing, but i think it's pretty obvious that if you make your loop bounds depend on the input you are asking for timing attacks
08:38 < adam3us> andytoshi: 256-bits (0..255) probably just for power 2 & word size divisibility.  not necessary  he says you can reuse the bit in bit-stealing if u want.
08:39 < andytoshi> oh, good to know
08:42 < adam3us> andytoshi: so the thing with the co-factor is there is  think a small subgroup also as n=h*l (l subgroup size, h cofactor, no points on curve), so trailing 000 is actually computing d=rand(0,l), Q=dG, Q'=8Q i think to avoid the small subgroup,
08:44 < adam3us> andytoshi: (ie the useful private key is 251 bits, and presumably |l|=251) but still u stay in the subgroup once u start there, so Q"=Q'+MAC(chain,ctr)G is still in the subgroup (hopefully!)
08:45 < adam3us> andytoshi: actuall i guess |l|=252, the top-bit is not "useful" as its fixed but its still part of the private key
08:47 < andytoshi> adam3us: understood, though i suppose i don't understand why this subgroup is so bad that we actually need to zero it out
08:47 < andytoshi> i guess fixing a representative of the coset is needed for determinism
08:48 < adam3us> andytoshi: actually take that back |l|=253, paper:  l is the prime 2^252+2...
08:50 < adam3us> andytoshi: maybe its fixable.  its just an obtuse way of saying d=rand(0,l-1) Q=8dG right?
08:50 < adam3us> andytoshi: so do the *8 part after HD derivation
08:51 < andytoshi> that's my understanding, and also i think it's just obtuse so that he can hold bit 254 fixed
08:54 < andytoshi> adam3us: at first glance, doing *8 after derivation would make HD work, i don't think the cofactor is a problem for us (though it weakens security by those 3 bits, and gmaxwell is pissed that djb considers this "better" than the ~1.5 bits our curve loses by fast parameter choices)
08:54 < andytoshi> but i don't see how we can hold bit 254 on
08:54 < andytoshi> though i guess we could grind through HD keys until we get one with bit 254 set :}
08:56 < andytoshi> (kidding!) anyway i've gotta run, good talking to you
08:58 < adam3us> andytoshi: Bernstein et al's style is obtuse in general :(  cleary if ou do Q=dG (d chosen rand(0,l-1)) then you do HD derivation Q"=Q+M(c,i)G, and finally Q"=8Q' then there is no guarantee that top bit is 1 as its mod l, but we dont care about that anyway
08:59 < adam3us> andytoshi: well with public derivation u have no way to know what the top bit of the private key is.
12:39 < maaku> farewell net netrality, we hardly knew ye
12:41 < nsh> :/
13:05 < gmaxwell> maaku: you don't have any great ideas on how to prevent incentive buggary with expensive validation do you?
13:10 < gmaxwell> andytoshi: you called it correctly on my complaints.  WRT cofactor there are two complaints, one it that there is the direct rho security reduction from the decrease in the order, compariable in magnitude to the rho reduction he dings secp256k1 on.  The other is that having a non-trivial cofactor is a necessary precondition for index calculus, though just a single extra distinct prime factor is probably no concern, all thing equal we ...
13:10 < gmaxwell> ... should prefer thing with a cofactor of 1.
13:26 < adam3us> gmaxwell: so do u know does the trailing 000 bits of d matter?  that also will be lost by HD derivation
13:28 < pigeons> andytoshi: is there an irc channel or bitmessage broadcast address or something anouncing pending coinjoin sessions?
13:32 < gmaxwell> adam3us: so long as you compute the derivation the right way you can preserve it.  Basically the scalar you compute in it has to also be a multiple of 8.
13:33 < gmaxwell> adam3us: because the order is prime*8 the sum of any two numbers which are themselves a multiple of 8 will still be a multiple of 8, thats what results in it being a subgroup.
13:33 < andytoshi> pigeons: no, not yet, i have just released a client and having trouble finding testers during my free moments
13:34 < andytoshi> i'll check out bitmessage tho, that'd be fun to set up
13:35 < adam3us> gmaxwell: but if you are working in the subgroup, the scalars are then mod l not mod n=8l?  or is B a generator of the full group?
13:36 < gmaxwell> oh darn. hm. I think you're right. obviously the basepoint isn't going to generate the full group.
13:37 < gmaxwell> then I don't understand anymore why the private key is constrained.
13:38 < adam3us> gmaxwell: well a base point could be generator of the full group, i think (if they chose it that way?); and that may explain the 8s that appear in the verification relationship perhaps.
13:54 < maaku> gmaxwell: what's the context of "expensive validation" - my script musing on #bitcoin-dev?
13:54 < gmaxwell> maaku: yea
13:55 < maaku> well in some of the applications i'm imagining it could be more efficient to validate a message signature than a transaction
13:56 < maaku> so, you could sign the transaction itself as a message, efficiently proving you have the inputs, and then get gray-listed if the actual validation fails
13:57 < maaku> e.g. the script is "if real-transaction then <complicated covenant code> endif <standard pubkeyhash script>"
13:58 < maaku> i would like a better method though
14:00 < maaku> you could require something like the above if the (explicit) instruction count is greater than some normal-use threshold
14:02 < maaku> pigeons: ;;cjs
14:02 < maaku> ;;cjs
14:02 < gribble> Coinjoin Status: There is no currently open session. Visit https://www.wpsoftware.net/coinjoin/ or http://xnpjsvp7crbzlj3w.onion/ to start one.
14:02 < maaku> andytoshi: but it'd be nice if there was an announcement when a new session started
14:05 < adam3us> gmaxwell: so i was musing an analogous argument to pegged side-chain security (cant inflate supply of main chain) could be used to introduce SNARKs + committed-tx or some variant of it in a zero-coin like zerotrust mixer on the main chain
14:06 < michagogo|cloud> Anyone have a link to andytoshi's cj client?
14:06 < adam3us> gmaxwell: or perhaps more simply, just make a zerocash snark as a reference example of a pegged-side chain (though i note even green put a disclaimer in his talk that this is a bit bleeding edge and could have problems)
14:06 < EasyAt> maaku: Couldn't I send a bogus TX that has a ton of operations to verify to chew through processing power?
14:07 < adam3us> gmaxwell: which seems kind of ironic (proposing to integrate zerocash in the pattern in which zeroin was proposed), now that zerocash is proposed as an alt.  (and I and Hal were more excited about moving zerocoin into its own alt)
14:08 < EasyAt> You would have to do a ton of ops before you realize the TX isn't valid
14:09 < maaku> EasyAt: yes, which is why as I said above you might require that the owner provide a quick-verifying signature over the transaction of the expensive inputs
14:09 < maaku> so you know the transaction came from him
14:09 < adam3us> anyone how big is the UTXO set if compacted now?
14:09 < maaku> and then gray-list the inputs if the validation fails
14:09 < maaku> adam3us: gettxsetinfo or something similar
14:10 < maaku> EasyAt: then it at least becomes expensive to perform DoS
14:11 < michagogo|cloud> [off]test
14:12 < michagogo|cloud> Oh, are the logs not live?
14:12 < EasyAt> maaku: What do you mean by gray list?
14:12 < maaku> e.g. only pay attention to transactions with inputs that have less than 20 instructions, *or* transactions enveloped with a less-than-20-ops signature for the expensive inputs
14:13 < maaku> gray list would be a list of inputs you no longer relay transactions for, maybe for a period of time or require higher fees
14:14 < andytoshi> michagogo|cloud: source is at https://github.com/apoelstra/cj-client
14:14 < andytoshi> michagogo|cloud: windows build at http://download.wpsoftware.net/bitcoin/cj-windows.zip
14:15 < michagogo|cloud> thanks
14:15 < gmaxwell> andytoshi: about 300 mbytes.
14:15 < gmaxwell> oops
14:15 < gmaxwell> adam3us:
14:15 < michagogo|cloud> "cj-windows.zip is not commonly downloaded and could be dangerous."
14:15 < andytoshi> gmaxwell: !!!! ;)
14:15 < gmaxwell> bitcoind  gettxoutsetinfo
14:15 < gmaxwell> { "height" : 280494, "bestblock" : "00000000000000024c41edbc27cb0d093b593a47030b886fade01f9d19b8047a", "transactions" : 2597060, "txouts" : 8350183, "bytes_serialized" : 293414423, "hash_serialized" : "ca53e5d3a59fc7a3dca134cce6942c2af5d85c2ce21d985c8b06526e795faf74", "total_amount" : 12262214.79395749
14:16 < gmaxwell> }
14:16 < andytoshi> michagogo|cloud: populism is not security, your browser uses faulty assumptions
14:16 < michagogo|cloud> andytoshi: I know
14:16 < michagogo|cloud> I wasn't ascribing any meaning to that thing
14:16 < michagogo|cloud> Just wanted to let you know Chrome was flagging it
14:16 < gmaxwell> what a shitty thing
14:16 < andytoshi> ok, good to know
14:16 < andytoshi> chrome should really be flagging windows..
14:16 < gmaxwell> I bet if you throw the same binary on github you get no warning.
14:17 < michagogo|cloud> btw, I assume it uses RPC?
14:17 < michagogo|cloud> Which calls?
14:17 < michagogo|cloud> (i.e. can it work on 0.8.6?)
14:17 < andytoshi> michagogo|cloud: listunspent, createrawtransaction, decoderawtransaction, signrawtransaction, getaddress, walletpassphrase
14:17 < andytoshi> i think those are fine
14:18 < gmaxwell> also gettxout
14:18 < andytoshi> oh, gettxout, dumpprivkey
14:18 < gmaxwell> you might want to use getrawchangeaddress  but I think its git-only.
14:18 < gmaxwell> perhaps try getrawchangeaddress and if it isn't there, use getnewaddress?
14:18 < michagogo|cloud> In about 7 minutes when my 0.8.6-compatible blocks and chainstate finish copying over I'll see
14:19 < andytoshi> gmaxwell: what is the difference?
14:19 < gmaxwell> andytoshi: change addresses get hidden in the transaction list. But perhaps not. actually nevermind that if you do that people will spazz.
14:20 < gmaxwell> though .. actually you really should have a feature to let the user specify recipent addresses for the CJ outputs. (Personally I send my CJ outputs to offline wallets!)
14:21 < andytoshi> gmaxwell: agreed, my original UI sketch had such a thing
14:21 < andytoshi> but it's hard to design a UI for that non-intrusively
14:21 < michagogo|cloud> andytoshi: Hm, it doesn't seem to be launching
14:22 < michagogo|cloud> The process is ther, but just sitting at 164K of memory
14:22 < michagogo|cloud> there*
14:22 < andytoshi> michagogo|cloud: any output?
14:22 < michagogo|cloud> and not visibly opening anything
14:22 < andytoshi> my guess is that it's stalled pinging my server..
14:22 < michagogo|cloud> Oh, that's why
14:22 < michagogo|cloud> I don't know why it took so long to show up
14:22 < michagogo|cloud> "Our information on this file is inconclusive."
14:22 < andytoshi> oh, weird, it's quick for me (and i'm a good 2500km from the server)
14:23 < michagogo|cloud> "We recommend not using this file unless you know it is safe."
14:23 < gmaxwell> well it does connect to the remote server at startup.
14:23 < andytoshi> oh fuck windows
14:23 < michagogo|cloud> andytoshi: Nah
14:23 < michagogo|cloud> Not Windows, security software
14:26 < adam3us> gmaxwell: so that is 275MB vs 13GB for utxo vs txo about 2%
14:27 < gmaxwell> more like vs 16G.
14:28 < adam3us> gmaxwell: oh i thought jgarzik said his torrent was 13G
14:28 < gmaxwell> adam3us: sipa did some charts a long time ago, utxo size looked to be ~log() the blockchain size.
14:28 < gmaxwell> the torrent doesn't take it up to tip.
14:29 < adam3us> gmaxwell: (sending email cc green re contact from the other crypto guy mentioned in PM, i thought I'd take the opp to correct his 16GB bitcoin vs 1.2GB zercocash claim;)
14:36 < michagogo|cloud> andytoshi: eww, always-on-top?
14:37 < nsh> you don't get anywhere in the dog-eat-dog world of windowing systems by ceding your platform
14:39 < andytoshi> michagogo|cloud: what is always on top?
14:39 < michagogo|cloud> The cj client
14:39 < andytoshi> really?
14:39 < michagogo|cloud> Yes.
14:40 < maaku> who doesn't like it on top
14:40  * michagogo|cloud 
14:40 < andytoshi> oh, oops, i had gtk_window_set_keep_above () in there
14:40 < andytoshi> i didn't notice because i don't use a floating WM
14:40  * gmaxwell xmoand user unaffected
14:40 < michagogo|cloud> ;;google xmoand
14:40 < gribble> [Arena PvP] Xmo and Xtk 2v2 - Forst Mage/Mage pt 1 - YouTube: <http://www.youtube.com/watch?v=jHdT36vjQN0>; Xmo and Xtk TCB Double Frost Mage 2v2 Arena Part 1 - YouTube: <http://www.youtube.com/watch?v=YFMWKyYmioY>; Xmo and Xtk 2v2 Act II Double Frost Mage 2v2 Arena Part 1 - YouTube: <http://www.youtube.com/watch?v=hmH8F2MiSog>
14:41 < gmaxwell> yea, srsly. y'all use a floating window manager? sucks to be you.
14:41 < jtimon> xmonad?
14:41 < andytoshi> ....and a thought gmaxwell had a floating WM :P
14:41 < michagogo|cloud> And
14:41 < michagogo|cloud> Ah*
14:41 < andytoshi> michagogo|cloud: thanks much for testing, you are the first person with a normal system to have done so
14:41 < gmaxwell> No, I use xmonad.
14:41 < andytoshi> i'll refresh the build
14:42 < jtimon> hehe, I tried some tiling VM but I left it due to a lack of time for config
14:42 < gmaxwell> (I was happy I didn't need to report problems with the tiling wm, I guess I know why now)
14:42 < jtimon> I will definetely try again though
14:42 < gmaxwell> jtimon: to configure xmonad is very simple.
14:42 < gmaxwell> You join #haskell and nice people do it for you.
14:42 < jtimon> I shouldn't had started with ratpoison, but the name was so cool
14:42 < nsh> senate judiciary hearing on NSA started 10m ago
14:42 < michagogo|cloud> andytoshi: Is there a way to cj on testnet?
14:42 < nsh> http://www.c-span.org/Live-Video/C-SPAN3/   http://www.judiciary.senate.gov/hearings/hearing.cfm?id=32caee8082f9297f0e7df6280b369172
14:42 < jtimon> the two I used more were i3 and qtile
14:43 < michagogo|cloud> ;;tcjs
14:43 < gribble> Error: "tcjs" is not a valid command.
14:43 < michagogo|cloud> ;;cjst
14:43 < gribble> Error: "cjst" is not a valid command.
14:43 < andytoshi> michagogo|cloud: yeah, there is a cjconfig.conf file
14:43 < nsh> (Cass Sunstein currently summarizing review panel findings)
14:43 < gmaxwell> nsh: what did they find?
14:43 < andytoshi> in cjclient/, wherever Bitcoin/ is
14:43 < michagogo|cloud> andytoshi: What's the URL for the testnet page?
14:43 < jtimon> gmaxwell: hehe, I doubt the #python nice people will config qtile for me
14:43 < andytoshi> michagogo|cloud: oh, right, i forgot the server needed to support it
14:44 < andytoshi> one moment, i'll put it up..
14:44 < michagogo|cloud> andytoshi: There doesn't seem to be a %appdata%/cjclient
14:44 < gmaxwell> "Classified. Classified, classified classified classified classified classified classified. Classified classified classified classified. Classified classified classified, classified classified classified classified; Classified.
14:44 < gmaxwell> "
14:44 < jtimon> funny video introducing qtile http://www.youtube.com/watch?v=r_8om4dsEmw
14:44 < nsh> gmaxwell, something like that aye
14:45 < andytoshi> michagogo|cloud: is it listing your coins correctly?
14:45 < gmaxwell> jtimon: this is why you have to use xmonad. The programmers in other languages have actual applications for their skills. Whereas with haskell the only thing they have to do is configure xmonad for people.
14:45 < michagogo|cloud> andytoshi: I don't have a mainnet client open yet
14:45 < andytoshi> oh, my cjclient/ dir is using  c:/users/apoelstra/Local Settings/Application Data/cjclient
14:45 < michagogo|cloud> Ah, looks like the copy finished
14:45 < michagogo|cloud> one moment
14:45 < andytoshi> i don't think that Local Settings/ should be there
14:45 < jtimon> gmaxwell: loool
14:45 < michagogo|cloud> andytoshi: XP?
14:46 < andytoshi> michagogo|cloud: wine
14:46 < michagogo|cloud> WineXP?
14:46 < andytoshi> uh, i dunno
14:46 < michagogo|cloud> Or even earlier?
14:46 < andytoshi> yeah, winecfg says XP
14:46 < michagogo|cloud> I know Wine lets you choose which version of Windows to pretend to be
14:46 < michagogo|cloud> And I know that Application\ Data is pre-Vista
14:47 < andytoshi> ok, i'll figure out the glib function to get %APPDATA%..
14:47 < andytoshi> bitcoin is in %APPDATA%/Bitcoin yes?
14:47 < michagogo|cloud> Indeed
14:47 < michagogo|cloud> (C:\Users\Micha\AppData\Roaming)
14:49 < michagogo|cloud> ;;blocks
14:49 < gribble> 280495
14:50 < michagogo|cloud> andytoshi: oh, btw, it's not showing my coins
14:50 < andytoshi> michagogo|cloud: yeah, it's looking for Bitcoin/ in the wrong place
14:50 < michagogo|cloud> Ah, I see
14:50 < michagogo|cloud> https://www.irccloud.com/pastebin/u9B6Gp0X
14:51 < michagogo|cloud> andytoshi: Ah, I see. Specifically, it's using %localappdata%
14:51 < michagogo|cloud> I guess it doesn't plant its folder unless it manages to find Bitcoin/?
14:52 < andytoshi> michagogo|cloud: it doesn't plant its folder unless something changes, yeah
14:55 < jgarzik> adam3us, gmaxwell: current bootstrap.dat @ block 279,000 is 14222116865 bytes
14:57 < michagogo|cloud> Hmm, I wonder how hard it would be to change linearize.py to allow it to be given an existing bootstrap.dat and have it detect the latest block in there
14:57 < jtimon> maaku, what were you thinking on getting from factor to add to joy ?
14:57 < sipa> ;;blocks
14:57 < gribble> 280495
14:58 < michagogo|cloud> (so you can point it at an older bootstrap, and say "bring this up to 279,000")
14:58 < michagogo|cloud> Oh, wait, this is the wrong channel :S
14:58 < jtimon> or nothing concrete?
14:59 < jtimon> I'm thinking that joy should be easy to merklize, no?
15:08 < andytoshi> michagogo|cloud: ok, i have refreshed the windows download to use %APPDATA% properly (and fixed the keep-on-top thing)
15:11 < michagogo|cloud> Hey, there're my coins
15:11 < andytoshi> :D
15:12 < michagogo|cloud> andytoshi: Wait, so all this does is shuffle your coins around in your wallet?
15:12 < andytoshi> michagogo|cloud: yeah, it doesn't do spends right now
15:12 < andytoshi> i'm not sure how best to UI that
15:12 < michagogo|cloud> Imitate the Bitcoin-Qt UI?
15:13 < michagogo|cloud> andytoshi: Not working
15:13 < michagogo|cloud> Syncing with joiner, session ID unknown
15:13 < michagogo|cloud> Join server: error setting certificate verify locations:
15:13 < michagogo|cloud>   CAfile: /usr/i686-w64-mingw32/sys-root/mingw/etc/pki/tls/certs/ca-bundle.crt
15:13 < michagogo|cloud>   CApath: none
15:13 < andytoshi> weeird
15:13 < andytoshi> what if you delete the libcurl DLL?
15:14 < michagogo|cloud> The program can't start because libcurl-4.dll is missing from your computer. Try reinstalling the program to fix this problem.
15:15 < andytoshi> <.< ok, i'll look into this
15:15 < andytoshi> thx
15:15 < michagogo|cloud> np
15:34 < andytoshi> michagogo|cloud: ok, i have stolen some DLLs from msysgit, can you try redownloading?
15:35  * michagogo|cloud scrolls up a bunch
15:44  * michagogo|cloud slaps explorer.exe around a bit with a large trout
15:45  * sipa suggests installing an operating system
15:45 < michagogo|cloud> sipa: Hmm?
15:46 < sipa> nevermind, silly joke :)
15:47 < andytoshi> sipa: lol, be thankful that somebody on this channel has a normal system to test with
15:47 < michagogo|cloud> andytoshi: You messed up again
15:47 < andytoshi> michagogo|cloud: what now?
15:47 < michagogo|cloud> Same message as when I renamed libcurl
15:47 < michagogo|cloud> Except that it's complaining about libcrypto.dll being missing
15:47 < sipa> andytoshi: true that
15:47 < andytoshi> michagogo|cloud: oops :} i forgot to put that one in the zip
15:48 < michagogo|cloud> (and it is, indeed, missing)
15:48 < andytoshi> ..no i didn't .. it disappeared
15:49 < andytoshi> michagogo|cloud: sorry about that, fixed, can you redownload?
15:50  * michagogo|cloud puts http://download.wpsoftware.net/bitcoin/cj-windows.zip at the bottom of the buffer
15:50 < nsh> could you statically link these libs, andytoshi?
15:51 < maaku> michagogo|cloud: not hard i would think, just scan backwards to find the last block
15:51 < andytoshi> nsh: probably, but i'd have the same problems as i do with bundling DLLs, plus the usual problems (no upgradability, etc) of static linking, plus potentially bad linking issues
15:52 < michagogo|cloud> ;;cjs
15:52 < gribble> Coinjoin Status: There is no currently open session. Visit https://www.wpsoftware.net/coinjoin/ or http://xnpjsvp7crbzlj3w.onion/ to start one.
15:52  * nsh nods
15:52 < michagogo|cloud> andytoshi: nope
15:52 < michagogo|cloud> Syncing with joiner, session ID unknown
15:52 < michagogo|cloud> Join server: SSL certificate problem, verify that the CA cert is OK. Details:
15:52 < michagogo|cloud> error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
15:54 < maaku> jtimon: nothing concrete - just pointing out that Joy is a pure, simple academic language
15:54 < maaku> which is very good for a consensus system
15:55 < maaku> but in my experience, there is usually a handful of select hacks which offend purists but greatly simplify real world usage
15:55 < maaku> and Factor is a practical language of the Forth tradition, so we should look at that to see if there's anything to borrow
15:55 < michagogo|cloud> andytoshi: I g2g to sleep now
15:55 < nsh> maaku, any good examples of Joy in use? (that i might find accessible)
15:55 < michagogo|cloud> (or at least to bed...)
15:55 < andytoshi> michagogo|cloud: ok, thanks for testing
15:55 < michagogo|cloud> Good luck.
15:56 < nsh> 'night
15:56 < maaku> and yes, Joy - or any concatinative language - should be trivial to Merklize
15:56 < michagogo|cloud> I may be able to test more of future days
15:56 < michagogo|cloud> on*
15:59 < maaku> nsh: any kind of covenant
16:00 < maaku> e.g. i issue MarkBTC which is an IOU with 1% interest, but attach a covenant allowing my to buy it back at any time for principle + interest
16:01 < petertodd> http://www.reddit.com/r/Bitcoin/comments/1v7ayg/revolution_in_bitcoin_privacy_stealth_addresses/ <- getting good feedback on stealth addrs on reddit
16:05  * nsh nods, opens tabs
16:06 < maaku> more generally, my musing on this started from trying to Etherium within the context of just changing bitcoin's scripting system
16:08 < nsh> to <verb?> Etherium?
16:10 < maaku> heh, to do what is trying to be done with Etherium within a (minimally extended) bitcoin
16:11 < maaku> e.g. turing-complete financial contracts
16:11 < maaku> coin covenants, etc.
16:12 < maaku> for example, with a couple of script extensions and re-enabled opcodes, petertodd could make mastercoin fully validating and spv-safe
16:12 < maaku> (using covenants)
16:13 < nsh> interesting
16:13 < nsh> any notes or discussion online?
16:14 < maaku> this is an idea only 12 hours old :P
16:15 < nsh> ah, cool :)
16:15 < gmaxwell> maaku: at least you won't end up in the sad situation of having created a stack based language without roll or rotate.
16:15 < petertodd> maaku: actually months old - I proposed it for fidelity bonded bank stuff ages ago
16:15 < maaku> gmaxwell: :\
16:16 < maaku> petertodd: yeah i figured you'd been working on this, based on our conversation
16:16 < maaku> petertodd: did that involve quines? I thought that part was new
16:17 < petertodd> maaku: nah, it needed quines too, and actually credit may go to gmaxwell come to think of it - I'd have to check my IRC logs
16:17 < maaku> ah i'll go read those threads then
16:17 < nsh> ooo quines
16:17 < maaku> nsh: that's how the covenant workss
16:17 < nsh> oh, fascinating
16:18 < petertodd> maaku: I *think* most of it was private conversation actually - wizards didn't exist back then
16:18 < nsh> can you(s) elaborate?
16:18 < petertodd> maaku: (this was almost a year ago now)
16:18 < gmaxwell> Yea, well, my 'invention' in the covenant thread is that you could produce a quine SNARK transaction, which is slightly surprising since you don't know the validation key for a snark until you've finished the circuit.
16:18 < maaku> mandate (some of) the outputs to have the same conditionals
16:18 < gmaxwell> Without the snark there are dumb ways to accomplish it.
16:19 < gmaxwell> With the snark it sounds impossible if you don't think about it from the right perspective.
16:19 < nsh> hmm
16:19 < maaku> /dumb/boring/
16:19 < gmaxwell> yea, fair enough.
16:20 < maaku> nsh: yeah actually the coincovenant thread is basically a listing of what you could do with a turing-complete script language and introspective builtins
16:21 < maaku> the snark is just a really cool addition
16:21 < gmaxwell> Yea, I think nothing there requires the snark except for efficiency.
16:21 < gmaxwell> might be good to add some examples that need zero knoweldge
16:25 < maaku> petertodd gmaxwell: btw didn't mean to take credit for this old idea. i thought nsh meant the benefits of using Joy
16:27 < nsh> i'm curious in general and specific :)
16:27 < petertodd> I'm curious if joy brings us any joy.
16:32 < maaku>  cdr=-\
16:32 < maaku> 6jm
16:32 < maaku> sorry
16:33 < petertodd> maaku: glad to see you have (formerly) strong passwords
16:33 < maaku> haha, toddler found my keyboard
16:33 < maaku> gmaxwell: well there are bounties. you'd need a zk proof to safely claim a sha256 collision
16:35 < maaku> you can even design a covenant which forces revelation if the coins are to be actually used
16:35  * petertodd says hi to little maaku
16:36 < sipa> cdr-=\   -> that's actually potentially valid C code
17:34 < pigeons> adam3us: I just saw https://github.com/atoponce/d-note uses hashcash to generate a token before you can submit
18:05 < jtimon> ok, so I need a name for the TC merklized extrospective scripting extension I just understood hours ago
18:06 < jtimon> otherwise "the new thing" is taken and I cannot learn or think about anything else new too me
18:06 < sipa> tc?
18:07 < sipa> extrospective?
18:07 < nsh> turing complete, no idea
18:08 < nsh> network-external inputs maybe
18:08 < jtimon> tc = turing complete
18:09 < jtimon> extrospective = you can reference the scripts in the outputs of future transactions, parts in them, and maybe also the current utxo and the block header
18:09 < petertodd> jtimon: that's a pretty good description IMO
18:09 < jtimon> something outside the script itself
18:09 < jtimon> thank you
18:10 < petertodd> jtimon: more than current utxo too, but likely committed data of some kind within (to be clear)
18:11 < jtimon> although joy is a new addition and not necessary for the idea I like joyScripts, although I also like quineScripts, and we could also just maintain coincovenants  (although not all uses use quines/covenants)
18:14 < jtimon> petertodd, you mean previous data in the chain? I guess it could work if people provide proofs to the miners, but for some reason I haven't found yet, that intuitively scares me
18:14 < jtimon> also I don't know any use et neither
18:14 < jtimon> *yet
18:15 < petertodd> jtimon: well, there's the model where it's proof based, referencing the prevblock hash, or you can have a model where miners are expected to actually have some set of data on hand. (that could take a lot of potential forms)
18:16 < jtimon> stateless validation is very attractive
18:17 < jtimon> I'm not sure what you mean by referencing the previous block hash
18:17 < gmaxwell> any stateful process can be reduced to a stateless one just by gathering up the state and presenting it as an input.
18:18 < petertodd> jtimon: IE, make your script take a proof in the form of a merkle path to the prevblockhash
18:18 < jtimon> petertodd: what kind of commited utxo are we assuming if any?
18:19 < petertodd> jtimon: could be a lot of forms, could be a committed MMR TXO too
18:19 < jtimon> I see, just one of them
18:20 < petertodd> jtimon: well, you can do both if you really want :P
18:20 < petertodd> jtimon: and actually, if you do expiration, both could make a lot of sense
18:21 < jtimon> well, I think expiration would be necessary for your TXI thing, but I don't know much about MMR
18:23 < jtimon> the advantages and stuff, I just read that once but I don't remember the motivation
18:23 < jtimon> I'm going to read again
18:24 < jtimon> but maybe a hybrid commited expired-TXI + UTXO would make sense too?
18:24 < petertodd> exactly
18:24 < jtimon> oh, I see
18:24 < jtimon> you use the MMR structure for the TXI ?
18:25 < petertodd> one interesting thing is that you probably want the PoW algorithm to be tightly coupled to some subset of blockchain data - perhaps the last year/GB of it - so a PoW on the UTXO set is an attractive idea
18:25 < petertodd> right, for long-term MMR works really well
18:26 < petertodd> note that when I say "UTXO" set that doesn't necessarily mean it the way you would mean in bitcoin - for some extrospective scripting consensus system your utxo set might mean a lot of things that may or may not be coins
18:27 < jtimon> to be honest, I'm thinking in freimarket's utxo
18:27 < petertodd> e.g. the absolute extreme you can take this idea is for the system to be essentially a key-value global consensus, where keys are H(script) and values the output of those scripts (basically)
18:27 < jtimon> with asset types, unique bitstrings...
18:27 < petertodd> yup
18:28 < petertodd> and mastercoin needs to look something like that if it's going to be useful
18:28 < jtimon> well, values also have refHeight for interest/demurrage and I guess some other minor details
18:28 < petertodd> right
18:29 < jtimon> why " you probably want the PoW algorithm to be tightly coupled to some subset of blockchain data"?
18:29 < petertodd> my extreme example, which I guess I could call MetaCoin, could be done such that the scripts themselves are what define consensus currency systems within MetaCoin
18:29 < petertodd> jtimon: because you want there to be incentive for miners to actually publish the contents of the blocks they mine, rather than just headers
18:30 < petertodd> jtimon: basically with stateless validation you can wind up with miners having no blockchain data at all, and then find out that only a single party has the data, and hence can assist others in creating transactions (or no-one has the data and the coin gets stuck!)
18:30 < jtimon> an interesting thing is that with unique tokens, you have effectively a per-asset namespace that you can use as generic key/value store
18:31 < petertodd> jtimon: yes, *but* that's only useful if either multiple values can be associated with a single key, or the keys are scripts
18:32 < petertodd> jtimon: see, you can view a decentralized consensus system's blockchain as a weird type of cryptographic accumulator - it's easy enough to create a proof that some tx-thing existed or didn't exist in that chain, but you must have blockchain data to update (and create) those proofs
18:32 < jtimon> but the holders could take care of keeping their data, no?
18:33 < gmaxwell> how can you keep data if miners aren't even sending you enough to update your copy?
18:34 < petertodd> gmaxwell: well, remember how with MMR TXO you can get transactions mined with the assistance of third-parties who create the txin proofs for you? of course, with the txin proofs, miners with no blockchain data at all can safely mine the txs
18:35 < petertodd> gmaxwell: hence, you can wind up with a system that appears to work just fine, until one day you realize only one entity has a copy of some or all blockchain data - even worse if you've got some sharded (U)TXO set scheme going on
18:35 < jtimon> gmaxwell I thought your part of the trie in which your data resides cannot be modified if not by you, maybe I misundertood something about maaku's updatable structure
18:36 < jtimon> I also don't understand this senstence "that's only useful if either multiple values can be associated with a single key, or the keys are scripts"
18:36 < petertodd> jtimon: yeah, but what forces miners to actually publish the content of blocks to other miners? nothing
18:36 < petertodd> jtimon: e.g. with my "one entity has a copy of the blockchain" example, miners could be just sending their blocks to that entity, but not to each other, and the system will appear to work just fine
18:36 < petertodd> jtimon: maybe that happens due to lazyness, maybe due to sybil attack, who knows?
18:37 < jtimon> they need to publish the new root of the trie, and they want other miners to believe them, so they will send all the proofs they used to update the tree
18:37 < petertodd> jtimon: in a sharded system, it means you can 51% attack some *subset* of the (U)TXO space, likely with less than 51% of hashing power
18:37 < gmaxwell> jtimon: your own coin could only be modified by you, but all the neghboring branches can be modified by the holders of 2^levels-up coins.
18:38 < petertodd> jtimon: nope. Miners will lose money if they mine invalid blocks, so we can trust them not too do that 95% of the time, and it's in your incentive to very quickly mine the longest chain so you're not wasting your time...
18:38 < petertodd> jtimon: and if tx's can provide proof that they are valid to include in a block, all the better!
18:38 < jtimon> you're trying to explain me the problem of relying on archive nodes
18:39 < petertodd> jtimon: or hell, imagine some scheme where we're using SCIP moon magic so that miners can prove their blocks *are* valid
18:39 < petertodd> jtimon: roughly speaking, but it's really even deeper than that
18:39 < jtimon> I thought that wasn't a problem with maaku's latest updatable utxo design
18:40 < petertodd> jtimon: no it is, it's just not as likely to be an actual problem as some sharded blockchain scheme.
18:40 < petertodd> jtimon: mainly I'm interested in solving that because I think it's an important part of making consensus schems more scalable
18:40 < jtimon> miner 1 receives all the proofs it needs from regular users to update from UTXOn-1 to UTXOn
18:41 < jtimon> he sends the mined block and all those proofs to all miners
18:41 < jtimon> I'm still missing the problem
18:42 < petertodd> it's simple: what forces him to actually send those proofs to other miners? they can mine just fine without them, and have incentives to skimp on doing proper validation
18:42 < jtimon> you said it yourself " Miners will lose money if they mine invalid blocks"
18:42 < petertodd> now systems where miners can mine pairs of blocks, one valid and one invalid, as gmaxwell has suggested, help here
18:42 < petertodd> jtimon: yes, which means we can trust them not too!
18:43 < jtimon> if you don't send me the proofs, I don't hash on top of your chain
18:43 < petertodd> why not? if I'm not hashing, I'm not making money 99% of the time where the block *was* valid
18:43 < jtimon> and because I want everybody to hash on top of my block, I send the proofs to every miner
18:43 < petertodd> I might as well take that risk
18:43 < petertodd> as I say, the incentive isn't as strong as you think
18:43 < jtimon> I prefer to mine block n-1
18:44 < petertodd> why? you'll make more money if you mine block n 99% of the time
18:44 < jtimon> ok, I don't trust your 99 but I get your point
18:45 < petertodd> remember, what the bitcoin sourcecode implements doesn't necessarily match what a rational miner will actually do
18:45 < jtimon> so miners would have the incentive to send fake invalid blocks to distract competition
18:46 < petertodd> right now that costs too much because you can only mine either a valid, or an invalid, block
18:46 < sipa> they have an incentive to send fake invalid blocks to 49% of their competition...
18:46 < petertodd> now, if the system allows you to mine both simultaneously, perfect!
18:46 < jtimon> so that 99 turns into a 1 and everybody is happy again
18:46 < petertodd> yes
18:47 < petertodd> and/or tie your PoW scheme to some subset of blockchain data, and the other miners simply can't mine on your block unless you give them the data, therefore they'll mine on block n-1 and eventually overtake you (unless you have 51%, but we're screwed there...)
18:47 < jtimon> what's the problem then? we only need miners to spam each other so that  they don't  trust new blocks without the corresponding proofs
18:48 < jtimon> oh, I see, I guess your solution is better
18:49 < petertodd> well, actually I think both solutions are good, and for different reasons: the "firedrill" one helps test fraud proof code and ensure it actually works, which is valuable in of itself
18:49 < jtimon> sorry, I forgot you were trying to explain me the problem that justified the solution but I forgot you had a solution
18:49 < petertodd> heh
18:50 < petertodd> now here's the next problem: suppose we decide to shard the blockchain, we'll say UTXO's starting with MSB=0 go on one side, MSB=1 go on the other
18:51 < petertodd> so that's basically two parallel chains, and and they timestamp each other (really forming a *timestamp* chain over both)
18:51 < jtimon> sorry, what was sharding again?
18:51 < petertodd> jtimon: shard as in how databases are split up
18:52 < jtimon> like partitioning the blockchain?
18:52 < petertodd> now miners only mine one or the other chains contents, but %100 of the hashing power goes to the timestamp
18:52 < petertodd> jtimon: yes
18:53 < jtimon> sorry again, what's MSB=0 ?
18:53 < petertodd> MSG=most significant bit
18:53 < jtimon> ok
18:54 < petertodd> now, suppose somehow one entity controls 95% of the hashing power on chain 0, and they just don't publish block contents, but *do* contribute to the overall timestamping hashing power
18:54 < petertodd> they can't attack the timestamp - they only have 40% of the total hashing power - but they can make it impossible for any transactions to happen on chain 0
18:55 < jtimon> sorry, I got lost here "now miners only mine one or the other chains contents, but %100 of the hashing power goes to the timestamp"
18:55 < petertodd> suppose then they stop their attack - you're left with a bunch of blocks that have been timestamped, but the actual contents of them have vanished, which means you can't modify the state of the chain unless you "roll-back" to whatever data is publicly available, but what's the right rule to handle that?
18:56 < jtimon> the purpose of sharding is to have lighther miners, I guess
18:56 < petertodd> jtimon: suppose a block header for the timestamp contains hashes of the most recent header in the subchains
18:56 < petertodd> jtimon: exactly, specifically spread the bandiwdth out so that you don't need to keep up with all tx's to mine
18:56 < jtimon> and who pows the timestamp?
18:57 < jtimon> nobody?
18:57 < petertodd> well, one easy way is to say that the two chains are merge-mined with the timestamp
18:57 < petertodd> and then set the pow difficulty to be exactly half of the timestamp difficulty
18:57 < jtimon> isn't merged mining like the opposite of sharding?
18:58 < petertodd> jtimon: no! in this case it's just a way of having a very strong pow for what orders transactions - the timestamp chain - while allowing for two separate chains
18:59 < petertodd> e.g. a block header in this scheme consists of PrevTimestampHash, MergeMineRoot, Time, etc.
18:59 < petertodd> and the subblock headers are just PrevSubChainHash, MerkleRoot
19:00 < jtimon> mhmm I don't know how something that is not powed can be very strong
19:01 < petertodd> jtimon: how is it not PoW'd?
19:01 < jtimon> oh, I see, each chain mines on top of the previous timestamp, not the previous block of the subchain, no?
19:01 < jtimon> what if one chain goes faster?
19:02 < petertodd> heh, well, interesting question!
19:02 < petertodd> you can probably come up with a scheme where the actual headers, just not the block contents, are known to both miners, and you adjust difficulties appropriately
19:03 < petertodd> but that's far from the most interesting part of this stuff
19:03 < jtimon> the most interesting part is having ligher miners, no?
19:04 < petertodd> jtimon: well, that's why you'd do it, but in the process you've made it succeptable to new attacks that didn't exist before
19:05 < petertodd> like I say, if the data for one chain isn't available for whatever reason, things get ugly, and less than 50% of total hashing power can attack the chain that way
19:08 < petertodd> one thing you can do is have "challenges": pick a nonce in the top timestamp chain, and make the rule be unless the data from that subchain turns up - along with an appropriate proof - the way you decide what is the best block changes such that you *can* reorganize that subchain with <50% hashing power
19:08 < petertodd> (normally you can't due to the timestamp property)
19:09 < petertodd> at least then if subchian data gets accidentally lost, somehow, the state of the system can recover.
19:10 < petertodd> that also somewhat protects you against malicious attackers, essentially because you can temporarily pay higher fees to get the rest of the miners to force some <50% attacker to spit up the data you actually need to make your transaction
19:10 < petertodd> and once you make a robust scheme with two subchains... it trivially extends to a full on tree
19:10 < jtimon> mhmm, I don't know...it seems very complex, maybe we just need to think about another way of sharding
19:10 < petertodd> it is very complex, but my suspicion is that sharding inherently is complex
19:10 < petertodd> just handwaving and assuming global consensus is *way* easier
19:10 < petertodd> pity it doesn't scale though
19:13 < jtimon> yeah, I've been thinking about other sharding-like schemes but for now they were broken (well, the first one actually just needs every node to trust each other, that is, is centralized)
19:13 < petertodd> heh, well doing it with trust is easy :)
19:13 < sipa> jtimon: nah, that's just called ripple
19:14 < jtimon> sipa: no, I don't mean ripple, I mean something scalable
19:14 < petertodd> jtimon: heck, just arbitrarily saying "OK! it's 8 block chains now!" will probably work in practice, even if really the security isn't as good as it could be
19:14 < petertodd> jtimon: ripple is scalable technologically, socially OTOH...
19:14 < jtimon> well, I haven't seen any centralized markets infinitely scalable
19:15 < petertodd> jtimon: ripple the idea isn't centralized
19:15 < sipa> i've mentioned it before, but i'd like to stop confusing the word 'centralized' with 'trust-free'
19:15 < jtimon> can the ripple.com network process 1 billion tx/s ? definitely no
19:16 < sipa> sorry, 'decentralized' with 'trust-free'
19:16 < jtimon> petertodd: 2PC ripple is compltely scalable
19:16 < petertodd> jtimon: exactly, ripple.com is an abomination and we shall not mention it again
19:16 < sipa> i shall resist.
19:17 < sipa> anyway, you could have a bitcoin-like system, where instead of script verification, there was just one huge datacenter computing zero-knowledge proofs of the validity of the chains
19:17 < sipa> it would be totally centralized (as in central point of failure), but to an extent trust-free
19:17 < jtimon> well, I think my centralized system is completely scalable, but maaku and I have to actually test that
19:18 < maaku> jtimon: another Joy-derived language that might be useful http://www.cat-language.com
19:18 < petertodd> sipa: which is roughly what my fidelity-bonded foo ideas were about, especially the fidelity-bonded ledgers version
19:18 < sipa> also, a bunch of N nodes all talking to eachother that all trust eachother is perfectly decentralized, but not trust-free at all
19:18 < jtimon> maaku I read that one has strong typing instead of dynamic
19:18 < maaku> yes, which would be a good thing i think
19:19 < petertodd> sipa: yes, and with some changes to the way blocks are structured you certainely could have groups of miners who trust each other co-operatively create and mine blocks with individually low-bandwidth nodes
19:19 < petertodd> sipa: I think you can even pull that off as a soft-fork
19:20 < sipa> anyway, i'm not making any particular suggestion here
19:20 < sipa> just trying to point out that 'decentralized' is ambiguously used in bitcoin context
19:20 < jtimon> well, I try not to use ZKP or snark/scip when designing, haven't learned black magic yet...
19:20 < petertodd> jtimon: if it can't be done with hashes, it's not really bitcoin
19:21 < jtimon> sipa: I get your point, sometimes just trust-less is enough
19:22 < jtimon> maaku: why do you think strong typing would be better? the little I read about joy sounded very good
19:23 < maaku> jtimon: enforced strong typing is less likely to result in consensus bugss
19:23 < jtimon> like if I could even like the language and all
19:24 < petertodd> maaku: are you thinking of including those languages as libraries or what?
19:24 < jtimon> I see
19:24 < jtimon> replacing the current scripting I think
19:24 < maaku> petertodd: currently hypothetical replacements for script
19:24 < sipa> any reason why you'd use a stack-based language, and not something ast-based?
19:25 < sipa> (i've been following about 1% of the discussions here the past weeks, i've certainly missed a lot)
19:25 < jrmithdobbs> rust txn scripts please!
19:25 < jrmithdobbs> ;p
19:25 < petertodd> maaku: right, because I was going to say, I think you're much more likely to avoid consensus bugs by just making the underlying opcodes/interpreter simple - screw what actual language you end up with
19:26 < petertodd> maaku: strong typing means you have types, and types themselves are a bunch of code with potential consensus failures
19:26 < maaku> sipa: it'd be (relatively) easy to transition from script to some other Forth-like language - essentially just write a translator between the two
19:26 < maaku> and it's nice that, like LISP, the syntax is simple enough that you can code directly and don't really need a compiler
19:27 < sipa> you can do the same for an AST like language
19:27 < sipa> well, in one direction at least
19:27 < sipa> and i'm sure it's much closer to the domain we're representing
19:28 < jtimon> maaku wasn't there more reasons to chose a concatennative lang ? http://en.wikipedia.org/wiki/Concatenative_programming_language
19:28 < petertodd> sipa: I think the big question is do you need the self-modifying code that forth makes possible?
19:28 < petertodd> sipa: for quines it's certainely useful
19:28 < sipa> quines? what do you need that for
19:29 < maaku> sipa: covenants
19:29 < sipa> another thing i need to read... :'(
19:29 < petertodd> sipa: IE things like SPV-verifiable colored coins
19:29 < petertodd> sipa: write a script that forces the transaction spending it to have a certain form, propagating the colored coin definition like a virus
19:30 < jtimon> basically, forcing the outputs of the next transaction to have certain code in their scripts
19:30 < jtimon> well, maybe only some of the outputs
19:30 < maaku> and Forth-like languages are really good for this sort of thing, although not required
19:30 < sipa> right
19:31 < maaku> since you basically just have to test the prefix of the output script
19:31 < petertodd> also, merklized abstract syntax tree schemes *are* very forth compatible, even the self-modifying quine versions
19:31 < jtimon> actually petertodd the colored coins example was confusing to me because of regular colored coins and freimarkets
19:31 < petertodd> forth is just symbol tables, and symbol's can just as equally be merkle hashes
19:32 < maaku> petertodd: re simplicity, that's why i'm looking at Joy/Cat. it's basically two dozen or so combinators + builtins
19:32 < maaku> and a syntax that is even simpler than LISP
19:33 < petertodd> maaku: incidentally, keep in mind that as complex as these sharded blockchain ideas are, they can also make these computationally intensive consensus schemes more viable by spreading the computation and space across more miners
19:33 < jrmithdobbs> forth has seemed like the obvious choice since i first saw the script ... first thing to come to mind was "why isn't that forth"
19:33 < petertodd> maaku: syntax has nothing to do with what goes in the chain necessarily... :)
19:34 < petertodd> jrmithdobbs: because satoshi didn't want a complex symbol table!
19:34 < petertodd> jrmithdobbs: script is even *simpler* than forth
19:34 < jrmithdobbs> petertodd: i know
19:34 < jrmithdobbs> petertodd: but forth is such a great fit for this use case ;p
19:34 < maaku> jtimon: yes i don't think the colored coin example is good for explaining the purpose, but IOU with a buy-back option is a good succinct example
19:35 < petertodd> jrmithdobbs: agreed
19:35 < maaku> i don't think satoshi realized that you could prefix an execution counter to the scriptSig to solve most of the turing-complete worries
19:36 < gmaxwell> sipa: petertodd pointed out that you can make colored coins where the network tracks the color for you by using a covenant scriptpubkey that basicially handles the task of making the network track which coin is colored.
19:36 < gmaxwell> maaku: I don't think a counter is sufficient for resolving "worries"
19:36 < petertodd> maaku: I'm sure he did, and thought of additional issues
19:36 < maaku> gmaxwell: i think the DoS preventions I mentioned in the scrollback solves the remaining worries
19:36 < maaku> is there something I'm missing?
19:36 < petertodd> maaku: though then again, counting sigops in un-exectuted scriptPubKeys was a damn stupid idea
19:37 < jrmithdobbs> petertodd: ya i was going to say, i think you're inferring too much credit there
19:37 < jrmithdobbs> s/in/con/
19:37 < petertodd> jrmithdobbs: indeed
19:38 < sipa> i don't think satoshi considered several of his changes (disabling opcodes, limited block size, counting sigops) as hard rules, just temporary anti-dos measures
19:38 < petertodd> maaku: the DoS preventions work even better when you do per-tx PoW schemes
19:38 < gmaxwell> maaku: engineering ones, like operation counting bugs creeping in when people implement faster execution engines, or sandbox escape when people implement faster execution engines.
19:39 < jtimon> petertodd how don't you hardcode the per-tx pow?
19:39 < petertodd> gmaxwell: the latter problem isn't specific to execution counters, heck, even the former isn't totally
19:39 < jrmithdobbs> gmaxwell: or emulating some other form of state/etc through some other trickery that would hurt everyone's head
19:39 < petertodd> jtimon: why would it be hardcoded?
19:39 < jrmithdobbs> gmaxwell: those are the really scarey ones imho
19:39 < maaku> gmaxwell: yes, well that's why I'd prefer a simple language with a minimal number of primitives and implementation complexity ...
19:39 < gmaxwell> petertodd: no but IP counting is actually much harder when you've implemented a tracing JIT.
19:40 < maaku> i think you could implement Joy/Cat with the same or less lines of code as current bitcoin script
19:40 < gmaxwell> (A significant fraction of all code execution bugs in firefox have been in the JITs)
19:40 < jtimon> patertodd is just the simplest scheme that comes to mind, just want to know what you had in yours, what's your diff filter?
19:40 < petertodd> gmaxwell: well, that's an argument against sophisticated scripting in general too...
19:40 < jtimon> /pater/peter
19:40 < gmaxwell> maaku: right but when people actually use it there will be a lot of pressure to replace it with a JIT. And a lot of room for bugs resulting in sandbox escapes and instruction counting glitches.
19:41 < petertodd> jtimon: well, *absolutely* simpliest is to say a block has a single tx in it :)
19:41 < gmaxwell> petertodd: it is, indeed. but a JIT for a non-turing complete language is FAR easier to make safe. Esp since it can work with dumb template matching much of the time.
19:42 < jtimon> I thought it was per-tx pow apart from block pow, the thing I'm more confortable with are "optional pow fees"
19:43 < petertodd> gmaxwell: that's nice, but it all comes down to "is programming scripts easy enough and fast enough to be practical?" especially when we're talking things like covenants
19:43 < jrmithdobbs> gmaxwell: is replacing the script being seriously considered or just a toy conversation?
19:43 < petertodd> jtimon: I think it makes most sense when the only pow is in tx's, although exactly what that'd look like is an interesting question
19:44 < sipa> jrmithdobbs: even gavin has mentioned it (though i'm sure he's thinking about much less exotic changes)
19:44 < maaku> jrmithdobbs: jtimon and I are seriously considering it in the context of trying it out on an altchain (freicoin)
19:45 < petertodd> maaku: and it's on my list of things that I need to research for MSC
19:45 < gmaxwell> jrmithdobbs: most of this is speculative conversation. The thing I'd want to replace it with isn't yet realistic to deploy.
19:45 < maaku> although given all the other more important stuff on our plate, it's still a very hypothetical conversation
19:45 < jrmithdobbs> gmaxwell: which is?
19:46 < jtimon> with covenants we could replace we could replace some freimarkets stuff and already have a use case that was actually a missing piece for p2p lending
19:46 < petertodd> maaku: well, for me it's a top priority
19:46 < sipa> if i was asked today to write a script language for bitcoin, i think it'd be an AST with slightly lower level crypto operations than bitcoin has now
19:46 < gmaxwell> using some form of ZK-SNARK instead of doing fancy things directly. (I'd still be in favor of improving things generally, e.g. M-AST)
19:46 < petertodd> sipa: mostly agree there, what's interesting is what types of data would that script have access too?
19:47 < sipa> i don't think byte arrays as data type is such a bad idea
19:48 < jtimon> gmaxwell sipa will an AST really be simpler for script coders?
19:48 < gmaxwell> (since I don't think it ever would make sense to use a SNARK to accomplish a 'simple'  (X and Y) or ((X or Y) and 2of3(Q,R,Z)).
19:48 < jrmithdobbs> jtimon: huh? of course it would
19:48 < maaku> gmaxwell: the SNARK would still have a language it understands though, right? (e.g. tinyram)
19:49 < gmaxwell> maaku: no. What I'd do is just implement a generic snark validation, and providing the snark verification key in the transaction.
19:49 < petertodd> sipa: that's not what I mean actually, I mean does the script have access to things like txins, txouts, blockchain headers?
19:49 < sipa> petertodd: for bitcoin, no
19:49 < jtimon> jrmithdobbs I mean simpler than Joy, not simpler than the current one
19:49 < sipa> petertodd: well, generalizing the hashtypes may be useful
19:50 < maaku> sipa: if you give the script access to the transaction, block header, and utxo data, a lot of interesting covenant-related stuff becomes possible
19:50 < sipa> but that's not really an interesting discussion - i'm being intentionally conservative here
19:50 < gmaxwell> maaku: it's pretty hard to write a compact script to do things with that access however.
19:50 < petertodd> sipa: yeah, where I'm really more focused on something you'd need for MSC, day job and all :P
19:51 < gmaxwell> It would be nice if people would write some hypothetical scripts.
19:51 < maaku> gmaxwell: hard to write a compact script? howso?
19:51 < petertodd> gmaxwell: that's also on my priorities list
19:51 < gmaxwell> E.g. we know that enabling xor or add on hash outputs gets us a bunch of things, and what we have to do to get those things.
19:51 < maaku> do you mean the interpretor, or the script(s)?
19:52 < gmaxwell> maaku: no, go write a script using the disabled opcodes that and a hypothetical PUSH_OUTPUT_N and PUSH_INPUT_N that achieves a non-trivial covenant.
19:53 < gmaxwell> it's pretty easy to end up with a painfully complicated script just to do something conceptually simple.
19:53 < petertodd> maaku: you ever written any assembler code?
19:53 < maaku> ok, you mean in bitcoin script, yes
19:53 < sipa> jtimon: an AST is really equivalent to a stack language where every operation only consumes the N last entries (with N known before evaluating them) and produces a single one
19:53 < maaku> petertodd: look at the scripts we have in the back of the freimarkets paper ... yuck
19:53 < sipa> and you have end up with a single output
19:54 < jtimon> sipa, yeah, this http://en.wikipedia.org/wiki/Abstract_syntax_tree
19:54 < maaku> that's why I'd like a more powerful scipting language - even with opcodes re-enabled it's still a mess
19:54 < gmaxwell> maaku: but the problem with more powerful is that as soon as you color outside the lines you're back to a mess.
19:55 < sipa> jtimon: yeah sure, just saying that it can't be harder to implement, it's pretty much a subset of what we have now
19:55 < jtimon> I meant that something like joy seems more powerful, thus simpler for the scripting language users
19:55 < sipa> scripting language users?
19:56 < sipa> i don't care about that - go use a compiler if necessary
19:56 < jtimon> the hackers writing bitcoin scripts
19:56 < gmaxwell> sipa: well there are complications, because we'd want M-AST but the merkelization should be optional... because it doesn't make sense to merkelize something which is smaller than the hash and which you don't need to keep private.
19:56 < jtimon> sipa: good point
19:56 < jrmithdobbs> gmaxwell: btw speaking of language semantic stuff ... please go yell at someone to implement higher order kinds in rust so that functor can be implemented correctly already ;p
19:56 < gmaxwell> Thats what an assembler is for. Plus for any real use of this you'll want an assembler with a theorem prover in it so you can actually know if your script works.
19:56 < sipa> jtimon: i care about easyness by which an implementation (script interpreter) can be judged to be correct
19:57 < maaku> sipa: is there a specific AST that is a good match to bitcoin?
19:57 < sipa> give me a day and i'll write you one
19:58 < gmaxwell> E.g. when eligius proposed using a multisig script for controlling their emergency pool recieving address the policy they decided they wanted was (X and Y) or ((X or Y) and NofM(Q,R,Z...))  and we wrote a script to achieve that... but we had no way to tell for sure that it was safe!
19:58 < maaku> i just mean i'm wondering if you had something in mine, like lisp/scheme or the spineless, tagless machiens of Haskell
19:58 < gmaxwell> and it wasn't so simple that you could just look at it and tell for sure it was safe.
19:59 < maaku> jtimon: btw theorem prover mentioned by gmaxwell is why you'd want static typing
19:59 < gmaxwell> (for _sure_ not just 'sure'... as it would potentially have hundreds of BTC assigned to it in a day)
20:00 < petertodd> gmaxwell: so these theorem provers, what kinds of languages don't they exist for?
20:00 < gmaxwell> so it would be nice if I could throw that into a theorm prover and ask it "is there any way to satisify this script that doesn't provide sixX or sigY"
20:01 < jtimon> oh, gmaxwell, I forgot that open problem
20:01 < gmaxwell> petertodd: the provers themselves are seperated from the languages, and there exist tools to convert code into inputs for them for a variety of languages.
20:01 < sipa> maaku: ok, there's a data stack that is initially populated with the script inputs (scriptSig data pushes), and a program, which is a serialized abstract syntax tree
20:03 < sipa> maaku: AST nodes are: access[i] (retrieves the i'th element on the stack, counting from the top, and returns it without modifying the stack)
20:03 < gmaxwell> In C I use a tool called frama-c that can drive a dozen different backend provers. (probably one of the best of the provers for proving things about program execution is http://alt-ergo.lri.fr/ )
20:03 < sipa> maaku: const[x] (just returns x)
20:03 < petertodd> gmaxwell: exactly, so I assume for non-turing complete AST's basically the provers are easier, but as you're saying, sounds like they exist for interpreted stuff as well - hence the desire to have a prover available is not directly a consideration for the language itself (at the consensus layer)
20:04 < jrmithdobbs> petertodd: they exist in compilers like he wants, even, for that matter
20:04 < jrmithdobbs> petertodd: (haskell does some forms of this during compilation)
20:04 < sipa> maaku: let(expr1,expr2), which evaluates expr1 and puts it on the stack, evaluates expr2 using that modified stack, and then pops the element again
20:04 < gmaxwell> petertodd: certantly the language design could influence how easy it is to have a prover. For C it's a complete cluster@#$@ and the provers are not terribly complete  E.g. there is no _sound_ prover for the full C language.
20:04 < sipa> maaku: and then some basic arithmetic/crypto/string/... whatever operators
20:05 < maaku> sipa: i see. thank you this helps
20:05 < gmaxwell> petertodd: because things like pointer deferences make life insanely hard for the provers. (though, they're not completely impotent)
20:05 < petertodd> gmaxwell: right, and again, this sounds like you're just getting pushed to what I've always thought is the most obvious way to do it: merklized forth
20:05 < sipa> maaku: you can map this indeed to a lazily evaluated functional language
20:05 < sipa> maaku: which would for example mean you don't need to evaluate expr1 in a let, if its expr2 never refers to it
20:05 < jrmithdobbs> also, I'm somehow watching 3 different conversations about this same basic subject in 3 different channels in the same window at the same time and I have no idea how that happened but it's confusing
20:06 < gmaxwell> yea, I do think that stack langauges can result in easy life for the prover, though there is still the free variable of how types work.
20:06 < maaku> sipa: i've been favoring strict over lazy due to implementation compleity and risk of consensus errors
20:06 < maaku> does the laziness gain you anything?
20:06 < sipa> speed
20:06 < jrmithdobbs> sipa: tbqh, a limited haskell98 with no IO monad would be perfect and without IO the runtime gets tiny
20:06 < gmaxwell> maaku: if we merkelize the untaken branches it also can get you improved privacy and reduced program size.
20:07 < jrmithdobbs> sipa: something like fay
20:07 < maaku> jrmithdobbs: more than that Haskell core language is quite simple and probably something worth looking at, or the even more low level STG machine in GHC
20:07 < jrmithdobbs> sipa: which is haskell98->js compiler sans typeclasses (so no monads period)
20:08 < gmaxwell> maaku: e.g. if your transaction can be redeemed way X or way Y   and Y is some 4of5 of 5 pubkeys   ... then you (1) save all that space in the transactions spending it way X, and also (2) don't disclose the details of way Y unless you use it.
20:08 < sipa> yeah, adding a choose operator that evaluates one of two branches, and takes a hash for the other, is easy enough to add here
20:09 < gmaxwell> One thing to keep in mind is that what we put in a scriptsig does _NOT_ need to be a copy of the program. What goes into the scriptsig is a _witness_ that proves the program was correctly evaluated. This doesn't require including the whole program.
20:09 < sipa> yup
20:09 < jrmithdobbs> ya that's the key part satoshi missed i think
20:09 < gmaxwell> This mental model obviously applies directly to the snark stuff, but it even works in conventional execution.
20:10 < jtimon> gmaxwell not with p2sh, does it?
20:10 < sipa> so, to elaborate a bit further
20:10 < gmaxwell> jtimon: we're talking about a generalization of P2SH that works recursively, effectively.
20:11 < sipa> you associate a hash with every ast node
20:11 < maaku> gmaxwell: what was that point about merkelizing in response to?
20:11 < gmaxwell> sipa: I still think you need to have non-hashing nodes in your AST, because its wasteful to hash for a single operation.
20:11 < gmaxwell> 17:06 < maaku> does the laziness gain you anything?
20:11 < sipa> gmaxwell: yup, that's easy to do
20:12 < jtimon> gmaxwell so you're saying that without the snark you need the merklization for what you want, got it
20:12 < jrmithdobbs> that question seems so backwards to me
20:12 < sipa> instead of associating a hash with every node, you only associate one with every "tree piece"
20:12 < maaku> oh i meant lazy vs strict parameter evaluation (e.g. Haskell)
20:12 < jrmithdobbs> after doing nothing but writing haskell for the last 2 months
20:12 < jrmithdobbs> lol
20:12 < sipa> tree pieces are delimited by choose operators
20:12 < maaku> yes you definately need lazy/short-cut conditionals
20:12 < petertodd> gmaxwell, sipa: remember that one potential way of doing this is rather explicitly with OP_EVAL and OP_HASH160 (essentially)
20:13 < gmaxwell> sipa: I think you could go further and have two kinds of choose operator, one that hashes and one that doesn't.
20:13 < sipa> gmaxwell: well there can be a regular ifthenelse operator
20:13 < sipa> that has no choose magic
20:13 < gmaxwell> right. fair enough.
20:14 < sipa> i'm saying the same thing i think
20:14 < sipa> except choose is special in that it explicitly takes a hash as argument, and not an expression
20:14 < gmaxwell> Right.
20:15 < petertodd> sipa: note that simple if-else-endif isn't sufficient if scripts or script fragments can return a value before reaching the end of the block - you might not want the rest of the block to be public
20:15 < sipa> but so is const or access, they don't take subexpression eithet
20:15 < sipa> petertodd: these are not imperative programs, there is no return operator
20:16 < sipa> they're just expressions
20:16 < petertodd> sipa: right
20:16 < gmaxwell> petertodd: even if there were you could always wrap hte hidden data with another choice.
20:16 < petertodd> gmaxwell: true
20:17 < sipa> yeah, choice is there to hide pieces of the script
20:17 < sipa> either because they are large
20:17 < sipa> or because they are private
20:17 < petertodd> sipa: hmm... so when is choice not something you can do with an if block?
20:19 < gmaxwell> (kind of a fun thing where we could make standard addresses  a choice with ecdsa in one branch and then a hash based quantum hard signature in the other... and if there is a compromise of ECDSA we soft fork to deny ecdsa redemption while people redeem coins via the hash based signing.)
20:19 < sipa> i don't think it's really an if in any caze
20:19 < sipa> let me come up with an example
20:19 < sipa> to do a 1-of-2 multisig
20:20 < sipa> let's say scriptA is something that fetches a sig from the stack and verifies it with pubkeyA
20:20 < maaku> hrm. I just realized that by executing code from the stack Joy/Cat makes it difficult to Merklize...
20:20 < sipa> scriptB is the same, but for pubkeyB
20:20 < petertodd> sipa: right
20:21 < petertodd> maaku: you can still merklize the initial code up to where the stack is executed
20:21 < jtimon> maaku: that seems right, I guess AST-script it is
20:21 < sipa> now you construct a script of the form choice(scriptA,scriptB), and put its merkle root in the output
20:21 < sipa> however, to spend it
20:22 < sipa> you either use choiceL(scriptA,hash[scriptB])
20:22 < sipa> or choiceR(hash[scriptA],scriptB)
20:22 < petertodd> sipa: see, I'm not sure how that's any different from IF <executed ops> ELSE <hash> ENDIF
20:22 < petertodd> sipa: which is how I always envisioned MAST to work
20:22 < sipa> it's an if then else, but the if/else is hardcoded
20:23 < sipa> it cannot be an expression
20:23 < sipa> its runtime semantics is just the identity
20:24 < sipa> it only affects how the hash of the script is computed
20:24 < sipa> note that choiceL(scriptA,hash[scriptB]) evaluates to just scriptA
20:25 < petertodd> right, and by that I mean in the binary representation of a script, you'd have some way to signify a IF code block that must never be executed, followed by the hash, vs. one containing actual opcodes
20:25 < sipa> right, but i don't like to think of it in term of executable operations
20:26 < sipa> it's just a tree with certain parts covered, by giving a hash instead
20:26 < petertodd> well, we're using similar words for the same thing :)
20:26 < sipa> sure
20:27 < sipa> but i think your original question really was
20:27 < petertodd> see, my real point is, with merklized forth it gets even more sophisticated, because your symbol table is hashes of code, and potentially at runtime you'd do something more sophisticated there just get some chunk of code dynamically
20:27 < petertodd> yet you can still arrange such that code that's never executed is never provided
20:27 < sipa> that's over my head :)
20:28 < sipa> anyway
20:28 < sipa> one question is if there are other merkle-choosing-like operations possible
20:28 < sipa> which do not mimick if-then-else
20:29 < sipa> i think if you have some for(i in [0..n], f(i)) operator
20:29 < petertodd> sipa: tl;dr: forth can do the magic that lisp can do, not with macros, but with self-modifying code
20:30 < sipa> with n a constant integer
20:30 < petertodd> right
20:30 < sipa> then you can have a merkle version of it as well
20:30 < sipa> that takes the hash of the non-evaluated loops
20:30 < petertodd> and for that matter, you can do tail-recursion for loops too...
20:30 < petertodd> and that can still be merklized
20:31 < sipa> without needing to reveal how many loops you wanted to be possible
20:31 < gmaxwell> sipa: well ... if you have a homorphic hash you can do 1 of N execution more efficiently. Though I'm not aware of any way to do that which we'd consider in scope for this discussion.
20:32 < sipa> haha
20:32 < maaku> petertodd: how are you going to merklize forth?
20:32 < maaku> ah, are you thinking of replacing a quoted block with its merkle hash?
20:33 < petertodd> maaku: remember, we're merklizing the potential code that can be run
20:34 < petertodd> maaku: so if you end up with code that defines new symbols, but doesn't use those symbols, then the symbol definition doesn't actually need to happen if that particular execution trace doesn't use them
20:35 < gmaxwell> sipa: so, linear iterative compression.
20:35 < gmaxwell> say you have some straight line code that can stop at some point.
20:35 < maaku> petertodd: ok, in Joy at least "if/else" is handled like so (I think it's the same for Forth): <predicate-evaluation> [quoted-true-block] [quoted-false-block] OP_IF
20:36 < maaku> in other words, push the code on the stack before execution
20:36 < petertodd> maaku: correct
20:36 < maaku> so I suppose we can replace the branch not taken with OP_RETURN (when executing), plus an affixed hash value for what was there
20:36 < gmaxwell> ins0 1 2 3 4 5 6 7 8        you compute  H(ins0....H(6|H(7|H(8))...)  and then if you execute and run to step 4 and stop, you'd provide  0 1 2 3 4 H(5...H(8)).
20:37 < maaku> ok that would work
20:37 < petertodd> maaku: and a symbol is a chunk of code, so you have <predicate> Symbol1 Symbol2 OP_IF, and symbol2 never executes, then where the symbol is defined in the first place can be replaced with just the hash of the opcodes that would have been put there
20:37 < gmaxwell> I think that structure is not equal to choices.
20:37 < sipa> gmaxwell: that's exactly what i meant
20:37 < sipa> with the for loop
20:37 < gmaxwell> okay, good then I came about to the same thought.
20:37 < gmaxwell> is there something that generalizes those two? are there more?
20:38 < sipa> very good question!
20:38 < sipa> but it's really about some parametrizable control flow
20:38 < sipa> oh um
20:39 < sipa> this is an expression language
20:39 < sipa> a for loop doesn't really make sense
20:39 < sipa> but you can replace it by a fold
20:39 < sipa> fold(3,f,x) computing f(f(f(x)))
20:40 < petertodd> sipa: you know, you can replace a for loop with repeated opcodes, and zlib compression...
20:40 < sipa> where that recursive hashing becomes much more apparent
20:40 < maaku> jtimon: see above ^^
20:40 < jtimon> yeah
20:41 < sipa> petertodd: that doesn't allow hiding the number of iterations from the root hash
20:41 < jtimon> "Combinators in Joy behave much like functionals or higher order functions in other languages, they minimise the need for recursive and non-recursive definitions."
20:41 < jtimon> maybe it's relevant although I'm starting to get tired and following your interesting conversation gets harder
20:41 < petertodd> sipa: ah, your example of a for loop is to loop based on a stack constant, not a symbol constant?
20:42 < sipa> petertodd: based on a constant given in the spending script
20:42 < petertodd> sipa: yeah, that's different
20:42 < sipa> petertodd: but NOT given in what goes in the root hash
20:42 < gmaxwell> fundimentally the _maximum_ depth of the loop could be hidden. (mean I can describe a language that allows this)
20:42 < petertodd> sipa: yup
20:43 < sipa> yes, you need to know a maximum iteration count
20:44 < sipa> but you don't have to reveal it
20:45 < gmaxwell> might be interesting to describe a hash based winternitz compressed signature in this language, assuming there exists an OP_PUSH_TX_HASH ... I propose that if our choice operator(s) are good then a maximally efficient winternitz signature will be completely natural.
20:46 < sipa> .. you lost me
20:47 < gmaxwell> sipa: you know how a lamport signature works, right?
20:48 < sipa> more or less, yes
20:48 < gmaxwell> for each message bit x, reveal either preimage_x or H(x) depending on if the message bit is 1 or 0. The public key is just the root hash over this data.
20:50 < sipa> hmm
20:50 < sipa> i need to see that on paper
20:50 < sipa> but now now
20:50 < gmaxwell> winternitz optimization:  take your message bits in groups of— say— 4 bits.  so your 256 bit message becomes 64  4 bits words.   you have then 64 preimages.  H( ... 16hashes total ..H(H(preimage_n)))  and your message word selects how deep in this structure you reveal.
20:51 < sipa> right
20:51 < sipa> so you weigh a smaller signatures over deeper hashes
20:51 < gmaxwell> that bit itself isn't secure, since someone could find another message where all the words had higher values than your message, but you can add a couple more checksum words, e.g. 3 for the case of 64 words. with their heights set to a sum of the others such that you can't reduce any of the message words without increasing at least one of checksum words.
20:52 < gmaxwell> in any case, that covers both the kinds of branching we've talked about.
20:54 < gmaxwell> so I suspect that if our language is done well, it actually reduces to one of these hash signatures... it's just doing some extra execution along the way. :P
20:54 < gmaxwell> or at least these signature schemes should express themselves very naturally in the script.
20:57 < gmaxwell> sipa: so here is another kind of 'choice': a choice where the permitted script is provided by the ScriptSig, validated by a key provided in the script and a checksig instead of a hash in the script.
20:57 < sipa> gmaxwell: actually, that fold operator can just be encoded using choice
20:57 < sipa> at every iteration you do a choice that just contains another instance of f
20:57 < gmaxwell> choiceL( choice(choice()))
20:57 < gmaxwell> yea..
20:58 < sipa> of course, if you provide a normal language construct for fold
20:58 < sipa> as an optimization
20:59 < sipa> you could provide a merkleizing one too
21:01 < gmaxwell> adam3us: So, is there a way with ECDSA, given three messages pick a pubkey,r,s  such that pubkey,r,s is a valid signature of any one of the three messages?
21:02 < gmaxwell> I guess pubkey,r,s isn't going to be smaller for just three. Alas.
21:02 < gmaxwell> (so much for my ghetto homorphic hash idea. :P)
21:19 < jcrubino> is there a workbook for bitcoin wizards in training?
21:22 < gmaxwell> No. I suppose I should make a references list?
21:22 < gmaxwell> a lot of the things we discuss have no references though.
21:22 < gmaxwell> E.g. I can't cite anything for merkelized abstract syntax trees.
21:30 < sipa> roconnor came up with those in an IRC pm with me :)
21:44 < petertodd> jcrubino: I keep threatening to write a book
21:44 < jcrubino> petertodd: I'll help
21:44 < jcrubino> Is it possible to download the dev mailing list from source forge?
21:45 < petertodd> jcrubino: I don't think so
21:45 < petertodd> how far back do you want?
21:47 < jcrubino> As far back as can be got
21:47 < jcrubino> I would like to do this: http://www.princeton.edu/~achaney/tmve/wiki100k/browse/topic-presence.html
21:47 < jcrubino> for the mailing lists
21:48 < jcrubino> tldr: topic modeling of the message contents
21:49 < petertodd> jcrubino: I've only got just under a years worth
21:50 < jcrubino> I could post to bitcointalk to ask for donations; but not sure how uniform they will be comming from different mail clients
21:51 < petertodd> jcrubino: well, test should be same I guess?
21:51 < petertodd> you can compare against different peoples copies by message id
21:56 < jcrubino> petertodd: how close to live release is mastercoin?
21:56 < petertodd> jcrubino: it has been released, for some value of "release"
21:56 < petertodd> jcrubino: there's live code out there that lets you move mastercoins around - is that useful however? good question
21:56 < jcrubino> true enough
21:57 < jcrubino> I was going to ask what is going to be the first real workd use case and then I remembered Willets original slide presentations
21:58 < petertodd> ...and what did you remember?
21:58 < jcrubino> A better question then is how far is bitcoin from 2.0?
21:58 < sipa> we're not even at 1.0...
21:58 < jcrubino> The good and the bad; he included it all
22:01 < jcrubino> sipa: will we have no idea what 2.0 will be untill we get there?
22:02 < sipa> i suppose
22:02 < petertodd> jcrubino: huh, interesting view of it... I'd say JR didn't include much at all, at least from what I remember
22:02 < petertodd> jcrubino: there will be multiple competing 2.0's is my prediction
22:02 < sipa> yeah
22:05 < jcrubino> ok wizards what is the most important thing to grok about bitcoin at the protocol level for wizards in training to be effective developers?
22:07 < sipa> i doubt wizards are a subser of developers
22:07 < sipa> *subset
22:07 < sipa> here it's much more about things that are cool to think about, beyond-bitcoin
22:07 < sipa> some that may be far from ever being implemented
22:08 < sipa> i see myself much more as a developer than as a wizard (mostly because of lack of time to keep up...)
22:09 < petertodd> sipa: agreed, and in the exact opposite situation personally
22:11 < andytoshi> jcrubino: the first reference i was given here was about random oracles, and that led me through a very enlightening reference chase:
22:11 < petertodd> jcrubino: I think the most fundemental thing I've discovered is the concepts of how mining can be separated into timestamping and proof-of-publication
22:11 < andytoshi> http://blog.cryptographyengineering.com/2011/09/what-is-random-oracle-model-and-why.html
22:11 < andytoshi> http://blog.cryptographyengineering.com/2011/09/what-is-random-oracle-model-and-why.html
22:11 < andytoshi> http://cseweb.ucsd.edu/users/mihir/papers/ro.html
22:11 < andytoshi> also see the fiat-shamir paper and 'probabilistic encryption' by goldwasser and micali
22:11 < andytoshi> if you can grok all those, that's enough background to ask intelligent questions re the crypto discussion
22:12 < andytoshi> petertodd: you have a writeup about this which i think is a very concise introduction to that idea
22:13 < andytoshi> i've lost the link and i didn't actually read it the first time, but that was my impression from the first few paragraphs :P
22:13 < andytoshi> s/concise/detailed
22:13 < petertodd> andytoshi: thanks
22:13 < jcrubino> thank you all, looks like some great reads
22:14 < petertodd> andytoshi: and I think that good writeup is also another important wizard lesson about Bitcoin: it's actually got very little to do with cryptography as you normally think of it
22:15 < andytoshi> petertodd: agreed, the regular crypto is necessary to banter about specific signature schemes (and to understand security models), but distributed consensus is its own field
22:17 < andytoshi> people here are very good at designing protocols which use bitcoin as a secure timestamp oracle, something i haven't quite got the hang of
22:17 < petertodd> andytoshi: yeah, and furthermore *decentralized* distributed consensus is it's own field again, notably a field where discussions of things like politics actually are relevant
22:19 < andytoshi> petertodd: yeah, i had a non-bitcoin-related political discussion earlier today and i realized that my naive libertarian beliefs have been greatly changed by discussion here about incentive structures in decentralized systems
22:20 < andytoshi> at that distance, i suppose it's just game theory, but decentralized distributed consensus systems give a very efficient model of this
22:20 < andytoshi> where a lot of the noise of human interaction is removed (by design) thanks to the trustless protocols
22:22 < petertodd> yup, and a very unforgiving model too, where you get to deal with relatively non-ideal participants
22:22 < gmaxwell> well when you spend a lot of time thinking in an adversarial model it changes how you think.
22:23 < gmaxwell> Normal thinking is strongly biased to thinking about the common cases, adversarial model thinking is biased to spend time thinking about the worst possible outcome.
22:25 < petertodd> which is what the non-wizards find so hard to deal with - witness the discussions of GHas.IO for instance
22:29 < gmaxwell> petertodd: I've found it interesting that people think there is no issue, then they get this "51% attack" idea in their head and think that like— if ghash.io gets 51% then suddenly all the bitcoins will be theirs— and then that misconception is removed and they're back to saying that there is no issue at all.
22:30 < gmaxwell> I guess this happens with all things.   Foo causes cancer! No it doesn't. Oh great! Everyone eat foo! Wait wait.
22:30 < petertodd> gmaxwell: suggests to me that people don't really understand the nature of the signatures in transactions, heck, likely they don't understand them at all
22:31 < CodeShark> most bitcoin users still probably believe that their bitcoins actually reside on their own computers and that addresses are where they are actually kept
22:32 < petertodd> CodeShark: oh, that's a good addition to the -wizards basic training list: understand semiotics and the distinction between sign, signified, and signifier
22:32 < gmaxwell> CodeShark: well the whole question of 'residing' deserves a Mu.
22:35 < gmaxwell> The answer "in the blockchain" is also wrong thinking— what happens if a blockchain is MMR compressed and only you have the data to prove your coins exists? Is it back in your possession now? What if that data has been further split into multiple parts with an error correcting code and spread to multiple machines. Now where does the coin reside?
22:35 < andytoshi> gmaxwell: oh, Mu is a very clean answer, thanks
22:36 < petertodd> gmaxwell: a good counter-question to that falicy is to ask people where does the song "Happy Birthday" reside exactly?
22:36 < andytoshi> gmaxwell: i spent an hour with a math grad the other day describing various cryptosystems and asking "where is the information stored"?
22:36 < gmaxwell> (and of course even ignoring MMR and whatnot wizards wank, it's kind of surprising to say something "resides" someplace public but can't be taken from there by anyone with access)
22:38 < gmaxwell> But also equally insane to say something like a coin resides with its private key, when the private key could be on a relativistic rocket and forever causually disconnected from any payments to it... :)
22:39 < gmaxwell> Fact of the matter is that we use analogies to understand thing by approximation. But there is no need that the (best) analogies need to be physically intutive, in fact basically all of higher mathmatics is about manipulating abstractions which are in no way physically intutive.
22:40 < petertodd> gmaxwell: also equally insane if you postulate an insecure signature algorithm that can be broken with 2^64 work
22:41 < petertodd> gmaxwell: I'll add to my wizards list that if you successfully got through a hard first year calc/analysis course with emphasis on proofs you're going to understand crypto-currencies much better
22:42 < gmaxwell> well, either that or it broke you completely and you're unable to reason without a pile of symbology in front of you.
22:43 < petertodd> heh
22:45 < petertodd> gmaxwell: probably a good thing I failed second year calc - the alternative was to be broken by it
22:46 < andytoshi> petertodd: second year calc is crap, it has no business being in a math degree
22:47 < petertodd> andytoshi: what did you do in second year calc?
22:47 < andytoshi> petertodd: half a dozen methods for computing second-year-calc integrals, and something about taylor series i think
22:48 < andytoshi> standard calc 2 fare, "here are some algorithms, run them by hand without ever checking hypotheses, hundreds of times"
22:48 < petertodd> huh, we did taylor in first year; second year was all about multi-variable versions of first year stuff, as well as a bunch of set theory stuff
22:48 < andytoshi> mathematical analysis was probably more difficult, but made far far more sense and was better motivated
22:49 < gmaxwell> I like that they don't even bother teaching people the chain rule in basic undergrad calculus anymore apparently.
22:49 < andytoshi> (analysis is when i switched into math honours and decided to take my degree seriously .. and also where i met my girlfriend :P)
22:49 < petertodd> gmaxwell: wtf?
22:50 < petertodd> andytoshi: typical, impressing a girl...
22:50 < andytoshi> gmaxwell: they do at UTexas at least ..
22:50 < gmaxwell> andytoshi: whew. okay perhaps I was just talking to idiots then.
22:51 < andytoshi> petertodd: that's pretty-much it, a bit ironic that now i'm in grad school 2500km away from her
22:52 < andytoshi> gmaxwell: i'd guess so, first-year calc only has 3-4 derivative rules plus a collection of limit stuff, there isn't much room to trim
22:52 < petertodd> gmaxwell: this is the second year calc curriculum that I took: Sequences and series. Uniform convergence. Convergence of integrals. Elements of topology in R2 and R3. Differential and integral calculus of vector valued functions of a vector variable, with emphasis on vectors in two and three dimensional euclidean space. Extremal problems, Lagrange multipliers, line and surface integrals, vector analysis, Stokes theorem, Fourier series, ...
22:52 < petertodd> ... calculus of variations.
22:52 < andytoshi> well, UT has room to trim, there's a ton of theoretical stuff that doesn't connect properly, so the course feels very rushed and confused
22:52 < andytoshi> petertodd: holy shit
22:53 < gmaxwell> andytoshi: I think this is because they don't teach differentation in algebra classes, but instead just teach people a bunch of rules which actually are differentiation, but don't explain why they work?
22:53 < petertodd> gmaxwell: first year was this: A theoretical course in calculus; emphasizing proofs and techniques, as well as geometric and physical understanding. Trigonometric identities. Limits and continuity; least upper bounds, intermediate and extreme value theorems. Derivatives, mean value and inverse function theorems. Integrals; fundamental theorem; elementary transcendental functions. Taylors theorem; sequences and series; uniform convergence and ...
22:53 < petertodd> ... power series
22:54 < petertodd> gmaxwell: well, almost, they changed the curriculum around and moved more of what I was taking into the harder class (that's the harder classes current description)
22:54 < andytoshi> gmaxwell: i think that's right, i explain to my classes what they are actually doing, and they (a) appreciate it and (b) act like they were completely unaware of it before
22:54 < petertodd>                   instead just teach people a bunch of rules which actually are differentiation, but don't 
22:54 < petertodd> andytoshi: heh, I don't feel so bad failing it then :)
22:55 < petertodd> andytoshi: the hard version of second year is this: Topology of Rn; compactness, functions and continuity, extreme value theorem. Derivatives; inverse and implicit function theorems, maxima and minima, Lagrange multipliers. Integrals; Fubinis theorem, partitions of unity, change of variables. Differential forms. Manifolds in Rn; integration on manifolds; Stokes theorem for differential forms and classical versions.
22:55 < andytoshi> petertodd: that's a serious calc sequence, what you listed was calc 1/2/3/4, two analysis classes, a variational calc class, and a bit of a third analysis class
22:55 < andytoshi> not that my school was terribly difficult, i spent the latter half of my undergraduate doing reading courses with professors instead of the standard sequence..
22:56 < petertodd> andytoshi: ha, lovely, and I did that after like six years doing a fine arts degree
22:56 < petertodd> andytoshi: that's UofT fwiw
22:56 < andytoshi> petertodd: wow, good to know then
22:56  * nessence wonders how many UT folks are @ cointerra
22:57 < andytoshi> i was told that grad students who go there teld to be unhappy, that the profs don't pay attention to them..maybe this is why
22:58 < petertodd> andytoshi: interesting - the teachers in first year weren't very good, and second year downright atrocious. You literally had TA's who were too shy to speak to students and spend the whole class facing the chalk board mumbling.
22:58 < jcrubino> petertodd: your my hero now with your calc story
22:59 < petertodd> jcrubino: lol
22:59 < andytoshi> petertodd: fwiw, at SFU we had very slow sequences as i described, then a serious problem with fourth-year students who had no knowledge of mathematics, but we had to give them degrees since we'd led them on for three years, and they'd be incomprehensibly stupid
22:59 < petertodd> andytoshi: ha! nah, uoft just fails people instead :P
23:00 < petertodd> andytoshi: first year calc we quite literally had about 90% of the class drop out
23:00 < andytoshi> yeah, that's an extreme workload especially if the teachers are all crap
23:00 < andytoshi> it'd be almost reasonable with excellent professors and TAs
23:01 < petertodd> yup, and a heck of a shock coming from art school - see, at art schools even the best in the field often take teaching jobs to earn some more money... I suspect with math it's a lot harder to attract talent on a budget
23:02 < andytoshi> yeah, generally there are rich schools who get ~2-400 applicants per year and get to choose -- then there are the rest who have perpetually open positions but terrible offers
23:02 < andytoshi> (UTexas is in the former, they accepted 30/400 applicants last semester o.O)
23:03 < petertodd> ha, uoft has 50k students
23:03 < petertodd> heck, they have more teachers then my previous school had students
23:03 < andytoshi> yes, it is very irritating when they make half the grad department TA calculus <.<
23:04 < petertodd> ugh, and it seems that uoft actually takes their better TAs and teachers and has them teach the easier math classes aimed at the non-math students
23:04 < andytoshi> 16k math students, 8k in the calculus sequences in any given semester, which means around 80 calc TAs i guess
23:05 < andytoshi> ( UTexas has similar total numbers to UofT i think)
23:05 < petertodd> probably about right
23:05 < andytoshi> petertodd: ugh, that's terrible
23:07 < petertodd> andytoshi: heh, well I still managed to learn enough from it to have some hope of learning more math :)
23:09 < andytoshi> very true, i was suprised to hear you (and gmaxwell) have so little formal math education
23:09 < andytoshi> i guess i don't really either, i have the papers but the upper-division part of my degree was almost entirely reading courses
23:10 < petertodd> gmaxwell is way ahead of me with math you know
23:10 < andytoshi> and i did a grad course in QFT, i had a fun time explaining to the physics chair who i was and what i was doing there :P
23:11 < gmaxwell> I don't know anything, but seeing as how I don't know anything I am also not afraid of anything.
23:11 < petertodd> decentralized consensus systems are probably the only "theoretical" branch of crypto I'll ever have a hope of coming up with new ideas in - notice how even my intuition for things like how ECC signatures work is relatively shakey
23:12 < petertodd> andytoshi: lol, quantum anything just sounds scary, and relatively useless :P
23:12 < andytoshi> petertodd: i did not notice that, your blockchain/MMR stuff is so advanced that i assumed you were a math/cs genius :P
23:13 < andytoshi> petertodd: quantum field theory is -very- shakey, it was a great course to learn what physicists are up to but i knew i didn't want to deal with it after that
23:13 < andytoshi> scott aaronson has somewhat changed my mind on that point tho, to the extend that he calls what he does "physics"
23:14 < gmaxwell> everything understandable to more than a few people is understandable to almost everyone if approached from the right perspective.
23:15 < petertodd> gmaxwell: +1
23:15 < andytoshi> gmaxwell: that's my feeling, i've managed to explain SNARKs on a conceptual level to people who haven't had any experience with crypto
23:15 < andytoshi> they have to let me talk for two hours about cryptography, though, so maybe i'm filtering people..
23:15 < petertodd> andytoshi: lol, I keep on thinking "what the fuck symbol am I supposed to use for foo?"
23:15 < petertodd> andytoshi: every time I try to write anything vaguely resembling a paper
23:16 < petertodd> andytoshi: were they smoking pot at the same time?
23:16 < andytoshi> petertodd: haha, that's the worst part
23:16 < andytoshi> petertodd: in one case, yes, he was the guitarist of a band at the bar near the math dept
23:16 < andytoshi> he kept getting ahead of me tho, and it turned out later that he had a tech startup 30 years ago and walked away with millions to play guitar..
23:17 < andytoshi> so i think he might've secretly known it all
23:17 < petertodd> andytoshi: it's so much easier in analog electronics... oh wait, I took MIT's second year course on that... :P
23:18 < petertodd> andytoshi: damn!
23:19 < petertodd> andytoshi: you know, it's interesting how many former comp-sci/physics/math/etc students you find in art school - I met easily two dozen, including ones that had pretty substantial degrees.
23:20 < andytoshi> interesting - in math departments, almost all the really good students are musicians, and some are even art school dropouts!
23:20 < petertodd> andytoshi: oh! I never met anyone with an art background at uoft
23:21 < petertodd> andytoshi: well, there was one cute girl who had been reluctantly pushed into biology by her parents in a physics class :P
23:21 < andytoshi> petertodd: nice :P
23:21 < petertodd> andytoshi: every day she dressed like she was going to some kind of anime con
23:22 < gmaxwell> to be fair, music may be far too practical for some mathematicians I've met…
23:22 < petertodd> andytoshi: although what was more sad I thought was the sheer number of med students in that class trying desperately to impress admissions to residency or whatever it is exactly :(
23:23 < andytoshi> petertodd: ugh, we had those at SFU too, it was so competitive and so sad
23:23 < andytoshi> but i like this talk of anime con girls, my gf took me to a con once and it there were a lot of them there :)
23:23 < petertodd> andytoshi: did you ever take any summer classes? this was in the summer...
23:24 < andytoshi> petertodd: yeah, one summer i took a variational calc class -- and joined the physics soccer team -- i met a few med students then
23:25 < petertodd> andytoshi: heh, what's funny about it is she was in first year, and I'll bet you had she gone to ocad like she wanted she would have quit that by second year in favor of being a hipster
23:25 < petertodd> andytoshi: exactly
23:26 < andytoshi> petertodd: yeah, in general i found the students at university should not have been there
23:27 < andytoshi> they'd really drag down the classes in later years, hence my retreat to graduate classes -- fortunately i was friends with enough faculty by then that there was no trouble with that
23:27 < petertodd> ah that sucks
23:28 < andytoshi> so my feeling is that uni in general is not a great decision if you want to learn things, better to meet the right people and work with them
23:28 < petertodd> yeah, I've never had any experience with the non-art university environment
23:28 < andytoshi> e.g. #bitcoin-wizards
23:28 < petertodd> yup, which is actually pretty much how my art school went - although the flip side of that is they've got a huge failue rate
23:29 < petertodd> but what's interesting, is the post-graduation outcomes, like employment, are surprisingly good, even compared to stuff you think is a good bet like engineering and medicine
23:30 < andytoshi> yeah, i've heard that -- and the art students i've met tend to be more open-minded and intelligent-seeming than a lot of math/science folks
23:30 < andytoshi> why the failure rate tho? do people give up or is it really difficult?
23:30 < justanotheruser> petertodd: are you an art major?
23:30 < petertodd> yup, and people skills goes a long, long way in our economy
23:30 < petertodd> justanotheruser: yup
23:31 < justanotheruser> petertodd: what do you do that is artistic
23:31 < petertodd> andytoshi: people find it hard to deal with the lack of structure is a part of it I think - art school has this weird existential dread to it where you know you have to keep on coming up with new art to succeed, but there's no magic solution to doing that
23:32 < petertodd> andytoshi: and you've got a very unforgiving social environment that's incredibly elitist, as it should be (though that's not necessarily typical - my school had a reputation for that)
23:32 < gmaxwell> in 1970 college enrollment was ~50% of highschool grads in the US, it's ~70% now.
23:33 < andytoshi> petertodd: that does sound scary, though i can see why the same sorts of people end up in math departments
23:33 < midnightmagic> i woukd be curious to know whether those include the new vocational colleges that have started calling themselves colleges..
23:33 < petertodd> gmaxwell: sounds like a bubble
23:34 < gmaxwell> midnightmagic: no.
23:34 < petertodd> andytoshi: well, to what extent are those departments like that as research becomes more of a focus?
23:35 < andytoshi> petertodd: it's directly correlated to research, the people doing plug-and-chug degrees weren't artsy at all
23:35 < petertodd> andytoshi: I mean, research can easily have that same kind of treadmill to it
23:35 < andytoshi> yeah, that's absolutely the way it seems to me
23:35 < gmaxwell> petertodd: well I believe that it was in the early 70s that the US government privitized sallie mae and made it so you couldn't discharge student loan debt in a bankrupcy.
23:35 < petertodd> andytoshi: ah, yeah, plug-and-chug has it's own kind of pain I think
23:35 < petertodd> gmaxwell: good point, and they're pretty good at recovering the money, eventually...
23:36 < andytoshi> gmaxwell, petertodd: the student debt situation is a bubble i think, but i've read the first few chapters of the bell curve (1994) which has some scary stats about school enrollment and IQ stratification which suggest that high enrollment rates are not all bubble..
23:37 < gmaxwell> petertodd: they only have to recovery a small portion of it if they inflate tuitions, made possible by tsunamis of money enabled by the lender favoring law...
23:37 < midnightmagic> yay flynn effect
23:38 < petertodd> midnightmagic: people who complain about "all those dumb poor people" mating with each other forget that if you're at the other end of the spectrum, having a smart partner is strongly selected for
23:38 < gmaxwell> I'm sure there are plenty of papers on largely debt based economies and the relative rates of bubbles in them.
23:39 < midnightmagic> petertodd: mm.. not so sure about that. engineers mating withe ngi eers appears to select for autism. additionally, genetic issues with ashkenazi
23:39 < petertodd> andytoshi: well, it could be both true that the high enrollment rates are perfectly justified, and it's a bubble, not to mention that the whole industry could change rather drasticly with these efficient online courses (that MIT EE class was great, and free...)
23:40 < andytoshi> midnightmagic: the bell curve talks about this, there is a real selection effect which appears to be increasing IQs amongst part of the population almost as fast as it is decreasing amongst other parts
23:40 < petertodd> midnightmagic: yeah, sounds like very strong evidence for selective mating to me...
23:40 < petertodd> midnightmagic: note that autism is a spectrum...
23:41 < andytoshi> s/almost as fast/faster, but amongst fewer people/
23:41 < midnightmagic> andytoshi: If the research I read is correct, they're destroying their own genetic viability longterm.
23:41 < andytoshi> midnightmagic: let's hope not, they're doing all the crypto research :P
23:41 < petertodd> midnightmagic: and speaking of, when I was a kid my mom had a tutoring/babysitting business that specialized in austistic kids - I ended up hanging around with a heck of a lot of them, and their parents, and it's remarkable how many were comp-sci/engineering
23:42 < andytoshi> midnightmagic: there has a general theme amongst human development of "intelligence overriding medical problems", so personally i am not too worried about such things
23:43 < midnightmagic> mm.. fairly confident that environment nutrition and opportunity are stronger than plain genetics in success. but i'm on my ipad so i can't build a cite frok my zotero library
23:43 < petertodd> andytoshi: that modern culture rejects racism so strongly also gives a chance for genes to be spread across large chunks of the population limiting the worst effects of all this stuff
23:44 < gmaxwell> midnightmagic: I've seen papers on that subject, environment/nutrition/opportunity are so correlated with the parents they're almost completely colinear, any paper that claims to control for that is basically just reporting on the outliers by definition.
23:44 < midnightmagic> (absent actual genetic problems anyway)
23:45 < gmaxwell> unless they plan on doing a controlled study, which has ethical problems.
23:45 < gmaxwell> (its not like we have enough twins data, especially enviromentally distinct twins data, to really say a ton about nature vs nuture)
23:46 < petertodd> midnightmagic: I'm pretty skeptical of anyone who tries to claim modern societies are all that bad at providing everyone with opportunities
23:46 < midnightmagic> gmaxwell: the last study i read from u of.. illi ois? compared like to like re: private and public schools and came up witht he remarkable conclusion that public schools on the whole educated better, while private schools benefitted significantly from the rich person advantages.
23:47 < midnightmagic> petertodd: I have a sociology department or two who would disagree witht hat :-)
23:47 < petertodd> midnightmagic: define "educated better"
23:47 < andytoshi> midnightmagic: right, and they're sitting comfortably in sociology departments instead of on the street, despite having no skills ;)
23:47 < petertodd> midnightmagic: and remember, I spent enough time at arts school to know sociology is often full of shit :P
23:48 < midnightmagic> petertodd: they measured that which was measurable: mathematics improvements between entry and exit grades for schools
23:48 < midnightmagic> .. lol now now. the research they do is better than forming opinions in a vacuum
23:48 < petertodd> midnightmagic: ah, so they measured improvements, who had the higher scores exiting?
23:49 < gmaxwell> petertodd: not all opportunities are equal to all. I mean sure, some child of some inner city gang bangers could have traveled 4 mi the the nearest library with internet access back in 2010, and joined #bitcoin and written some bad poetry for a thousand bitcoin and be a millionaire now.  But none did.
23:49 < midnightmagic> petertodd: public schools had grrater improvements when co paring identical students with identical backgrounda.
23:49 < petertodd> midnightmagic: just as easily you can explain that as private school kids started off smart and couldn't be educated much farther, or more importantly, they had better things to do with their time than focus on math improvements
23:50 < midnightmagic> petertodd: nope. they selected those ones out
23:50 < justanotheruser> midnightmagic: are you saying that schools should be segregated by income?
23:50 < gmaxwell> justanotheruser: they are already segregated by income.
23:50 < midnightmagic> as much as it's possible to know such things, there's now basically zero reason to think private schools provide a better education
23:50 < justanotheruser> gmaxwell: they bus students from poor schools to rich schools in some states
23:51 < midnightmagic> justanotheruser: nope. i'm saying private schools are lying when they claim to provide superior education
23:52 < petertodd> gmaxwell: heh, well, OTOH I know a guy from nairobi who did something not unlike that... moral of the story is raw opportunities actually don't do much in the face of culture and parents, and those are likely strongly geneticly related in many ways anyway.
23:53 < gmaxwell> okay, sure, I was also binning culture and parents in with opportunities. It's not like its your fault what parrents you had.
23:53 < midnightmagic> yes. adoptions help a lot with those kinds of studies too.
23:53 < midnightmagic> pretty fascinating how much people seem to be screwed if born to poor parents.
23:54 < petertodd> gmaxwell: yup, my point being blaming "society" for that kind of thing is misguided - we already do a tremendous amount
23:54 < gmaxwell> ::shrugs:: part of creating an optimally successful society is providing the infrastructure that helps people achieve their capability even if they're born into a dysfunctional family (and help family dysfunction not exist).
23:54 < petertodd> midnightmagic: gee, might have something fundemental going on...
23:55 < midnightmagic> petertodd: yeah, probably not straight genetics. some is, but parentage makes up for a lot of that. i.e. the success breeds overconfidence false loop
23:55 < petertodd> gmaxwell: yup, and frankly I *do* think we do a very good job of that and it's hard to figure out how to actually do a better job of it in most situations. I also think our effects, especially in schools, to further level the playing field are counter-productive - e.g. closing gifted programs in favor of yet more money at the lowest scoring percentile.
23:56 < midnightmagic> imo that kind of nonsense is b-s
23:56 < midnightmagic> closing gifted student programs?! wtf
23:57 < gmaxwell> well you are in canada. So perhaps things are better done there. :)
23:57 < midnightmagic> i'll let you know. i personally appear to be one of those weird outliers.
23:58 < gmaxwell> midnightmagic: "no child left behind" (a 2001 piece of education legislation and the resulting programs) in the US is often wryly refered to as "no child gets ahead"
23:58 < petertodd> midnightmagic: well, that's how the politics of it works. I know the people running the program where I lived then fought for years to keep it open, and always had to be very careful as to how it was portrayed - specifically they stressed heavily how the kids who were in the program statisticly did *worse* than the general population for a lot of different metrics, such as university admissions.
23:58 < midnightmagic> lol
23:59 < petertodd> midnightmagic: basically anything to avoid looking elitist
23:59 < midnightmagic> s art kids need a challenge or their study habits are nonexistent. yeah that makes sense what you're saying.
23:59 < gmaxwell> Se also: http://en.wikipedia.org/wiki/No_Child_Left_Behind_Act#Effects_on_school_and_students
23:59 < gmaxwell> er see*
--- Log closed Wed Jan 15 00:00:03 2014