00:40:24AdrianG:hi gmaxwell
02:15:21instagibbs:hearn: please don't discriminate against self-driving cars! They have (blockchain) rights too.
03:40:42Pasha:Pasha is now known as Cory
06:32:12petertodd:SF wizards: I'll be in SF this saturday, and I'm turning 30...
07:13:52brisque:"I think NaS has basically been solved."
07:16:19brisque:"Long range you use checkpoints/weak subjectivity, short range you use security deposits - doesn't really seem like an issue any longer IMO.'
07:16:47brisque:that is to say, proof of stake is totally workable if you allow complete centralisation.
07:41:04sipa:petertodd: my condolences; i turned 30.5 yesterday
08:05:17barjavel.freenode.net:topic is: This channel is not about short-term Bitcoin development | http://bitcoin.ninja/ | This channel is logged. | For logs and more information, visit http://bitcoin.ninja
08:05:17barjavel.freenode.net:Users on #bitcoin-wizards: andy-logbot fnxTX arubi justanotheruser lclc_ coiner hktud0 wallet42 Dr-G orik gribble p15x fanquake RoboTeddy MoALTz__ p15 dc17523be3 Cory antgreen adam3us nuke_ Adlai` LarsLarsen spinza huseby jcorgan x98gvyn cryptowest_ ajweiss melvster Emcy waxwing PaulCapestany berndj binaryatrocity Luke-Jr kinlo GAit OneFixt Logicwax kyletorpey midnightmagic mkarrer prodatalab Starduster Pan0ram1x devrandom crescendo stevenroose wizkid057 maraoz bosma
08:05:17barjavel.freenode.net:Users on #bitcoin-wizards: grandmaster harrow SDCDev otoburb copumpkin wumpus jaekwon sneak GreenIsMyPepper phantomcircuit jgarzik BlueMatt luny jaromil espes__ dgenr8 Apocalyptic gwillen dasource alferz bbrittain fenn hashtag_ amincd HM2 prepost tromp ahmed_ eordano nickler Alanius face PRab BananaLotus guruvan ryan-c lnovy DoctorBTC sipa amiller ebfull cluckj brisque cfields sdaftuar veorq coryfields helo Hunger- xabbix burcin runeks null_radix bedeho gmaxwell
08:05:18barjavel.freenode.net:Users on #bitcoin-wizards: SwedFTP epscy nanotube andytoshi bliljerk101 starsoccer comboy nsh Taek iddo EasyAt maaku btcdrak NikolaiToryzin s1w Visheate BrainOverfl0w roasbeef phedny so hguux__ MRL-Relay azariah btc___ throughnothing @ChanServ brand0 davout NeatBasis mr_burdell d9b4bef9 a5m0 Anduck CryptOprah leakypat TD-Linux K1773R indolering warptangent veox Eliel Graet jessepollak dignork lechuga_ Iriez AdrianG warren gnusha heath wiz jbenet mappum eric kanzure
08:05:18barjavel.freenode.net:Users on #bitcoin-wizards: petertodd JonTitor catlasshrugged Keefe Oizopower platinuum Muis michagogo Krellan catcow mariorz kumavis Zouppen artifexd yrashk Xzibit17 smooth airbreather isis dardasaba Fistful_of_Coins morcos [d__d] dansmith_btc yoleaux cursive Meeh fluffypony optimator livegnik
08:21:37nsh:30 is a great age to be
08:22:31brisque:it's not prime though. sipa is on the right track with almost-31.
08:22:37nsh:* nsh smiles
08:26:55fluffypony:* fluffypony is 33 in July
10:21:07Adlai`:Adlai` is now known as adlai
12:09:13c0rw1n:c0rw1n is now known as c0rw|away
12:22:08fanquake:fanquake has left #bitcoin-wizards
12:54:46instagibbs:never trust anyone over 30
12:55:03kinlo:right, that makes sense
12:58:58instagibbs:I'm under 30 so you can trust this statement kinlo
12:59:24kinlo:I'm pretty sure you're an idiot :)
13:03:07fluffypony:kinlo: *ur
13:15:03sipa:instagibbs: does 30.5 count as over 30?
13:15:04sipa:are you using natural or real arithmetic?
13:17:01instagibbs:sipa: ill bring that up at the local under over 30 meeting
13:22:03sipa:also, what number base?
13:31:04brisque:sipa: hex. you're really 48 in decimal.
14:57:45jtimon:* jtimon got stuck trying to convert 0.5 hex years to months...
15:02:48adlai:1825/16 days :P
15:12:16wallet42:wallet42 is now known as Guest21119
15:12:16wallet421:wallet421 is now known as wallet42
18:29:41luny`:luny` is now known as luny
18:33:41petertodd:instagibbs: are you over or under 30?
18:35:06petertodd:brisque: meh, the centralization that long-term checkpoints introduce may prove to be less damaging than the centralization ASICs and mining pools introduce; we don't have the perfect PoW implementation that we wish we had
18:40:40kanzure:petertodd: any opinions on the "lower the max block size" concept? :P ("so that we can see what happens when max block size is regularly hit in a somewhat more controlled fashion (without the entire network blowing up due to bandwidth/hdd constraints)")
18:41:29petertodd:kanzure: we already did that with the default soft-limit, and hearn and others campaigned heaviliy to raise it well before it got interesting
18:41:50kanzure:i am not sure i know about the soft-limit, whether it was implemented, and what the results were
18:42:05Luke-Jr:kanzure: it was lifted before it was relevant
18:42:26petertodd:Luke-Jr: ^
18:42:30petertodd:er, I mean
18:42:32petertodd:kanzure: ^
18:42:38Luke-Jr:dropping it back would be helpful, but good luck getting the monster pools to do it
18:43:17petertodd:pools are actually pretty receptive to dropping it, but the PR of doing so is bad on reddit :/
18:43:44kanzure:seems like we should be pushing for experiments that seem to be less harmful in general
18:44:27kanzure:the network might get stuck in other interesting ways with lower block size limits though, like maybe mempools explode and barf over everything
18:45:15petertodd:kanzure: well, it's easy to do the math on "mempools barfing" - just a function how many $/gb used you want to spend
18:47:17kanzure:i wonder if there was anything that could have been learned from that soft limit anyway. like "yes more devices can participate", but we already know that sort of.
18:48:19petertodd:kanzure: would have driven the adoption of non-chain tech faster
18:48:41petertodd:then again, we already saw that with changetip, gambling sites, etc.
18:49:07justanotheruser:reddit gets mad at pools for not having large blocks?
18:50:09Adlai:reddit gets mad at dissent
18:50:33justanotheruser:would they prefer the pools fill the coinbase with image macros?
18:51:09petertodd:justanotheruser: of course they do - it directly leads to transactions being more expensive, and larger blocks have no immediate downsides for SPV users
18:51:59justanotheruser:what benefit does more expensive transactions have for reddit?
18:52:37petertodd:justanotheruser: er, that's what I said... :)
18:53:25petertodd:justanotheruser: I mean, obviously there is no benefit to more expensive transactions for reddit because there's no benefit to smaller blocks for reddit - they're all SPV or hosted wallet users
18:54:13justanotheruser:oh, I was looking at that backwards
18:54:27justanotheruser:that actually sheds some light on reddit pushing super hard to increase block size
18:55:09petertodd:that's why we keep saying it's a tragedy of the commons situation...
18:55:47kanzure:i wonder if there's a way to construct proof or evidence of a certain amount of participation in blockchain consensus activities (not mining, though)
18:56:03kanzure:eh nevermind. i will formalize that later.
18:56:19petertodd:kanzure: how much do you want to bet that the answer will be basically no? :P
18:57:18kanzure:if nodes are not anonymous, they could sign consensus activity receipts or something, as a way of monitoring this?
18:57:23Adlai:the best i've seen (for mining) is to mine on p2pool, which 'proves' not only your hashpower, but how much you actually earned... using this for verifiable mining contracts is left as an exercise to the miners
18:58:36petertodd:kanzure: note how everything you're proposing adds a whole lot of complexity that will get rejected by the reddit crowd anyway "It'll all magically work! Incentives and stuff!"
18:58:47kanzure:oh, this is not for them, this is for my own benefit
18:59:04petertodd:ultimately I think the problem is we have people who aren't security engineers proposing to change a security-related system
18:59:19CoinCadence:Adlai: agreed.
18:59:39justanotheruser:does bitnodes check whether you're relaying stuff or just whether you accept incomming connections?
18:59:42CoinCadence:Just wish we could solve P2Pools scalability problems
18:59:55justanotheruser:if it's the former that's probably the closest you can get to that
19:00:13kanzure:the receipts i proposed would maybe be a solution to problems regarding hosted wallets and gossip/consensus participation (again, not mining) for reviewing whether nodes are rejecting your activity or something
19:00:18gmaxwell:CoinCadence: huh? I dunno what you're talking about wrt p2pool. It has basically perfect scalablity.
19:00:43CoinCadence:Smaller miners get squezed out as the pool rate increses
19:00:49Adlai:it doesn't scale down too well once the p2pool share difficulty gets high relative to ... what CoinCadence said
19:01:15gmaxwell:CoinCadence: Thats a misunderstanding.
19:01:52CoinCadence:5-6 big miners can drive difficulty very high, so smaller miners can not keep a share in the chain for when a block is found
19:02:10gmaxwell:If you'd said "solve variance" I agree there are things to improve there. But p2pool pays fairly in the expectation, regardless of the hashrate or hashrate ratios.
19:02:15CoinCadence:and variance gets very high...
19:02:39petertodd:sigh... solving varience is critical if we want people using p2pool - varience has a very real business cost
19:02:48gmaxwell:CoinCadence: Thats incorrect. :( the additional hashpower _decreases_ variance, yes, share variance goes up, but block variance goes down, and the net increase is a _reduction_ in variance.
19:03:06CoinCadence:not for the average home miner...
19:03:15Adlai:share variance is all that matters when your expected number of shares per block reaches 0
19:03:27gmaxwell:Basicallly p2pool gives at best a constant fraction redunction in variance.
19:03:29Adlai:* Adlai was in this situation for the last ~month of his miner's life
19:03:45petertodd:p2pool will never have as low variance as just using GHash.IO, and there just aren't good reasons to use it for the people who would use GHash.IO - running a full node has a very real cost
19:04:25CoinCadence:we have plenty of 0 zee public nodes available, they are underutalized
19:04:30gmaxwell:Adlai: That _NOT_ correct. holy crap, please go wrie a simulator and check it out. It doesn't matter _where_ you variance comes from. You are not better off "having a share" when there is no block found at all, that the pool sometimes finds blocks which don't include you is not a benefit to you vs not finding the blocks at all. :)
19:06:19petertodd:CoinCadence: yeah, for pools of any kind, p2pool public nodes included, marketting matters tremendously
19:06:20Adlai:wouldn't p2pool be better off with another large miner's worth of small miners in a hierarchical sharechain?
19:06:29kanzure:hmm those receipts would be useless because sybil, anyone can sign your messages and pretend to be a different node claiming they totes agree with your messages
19:06:55CoinCadence:I guess my point is it would be nice to coome up with a method of including more miners, with lower share vaiance without creating dust txs in the generation tx...
19:07:12Adlai:let them eat dust, as long as they mine it
19:08:04gmaxwell:Adlai: No, you can't have unbounded dust; existing hardware is often broken with large coinbase transactions, and huge coinbases would risk increasing orphaning rate if not for all the busted hardware out there.
19:08:23Adlai:gmaxwell: to 'share variance is all that matters', add 'for humans making irrational decisions based on expected utility'
19:08:36Adlai:which unfortunately seem to be the majority of participants in bitcoin
19:09:22gmaxwell:There are a lot of misunderstandings, some of these can be improved via better presentation and education. Twiddling the technology is almost never a good alternative to presentation or education.
19:09:33Adlai:* Adlai needs to mine again, so he'll have skin in this game
19:09:43gmaxwell:That the varance reduction is bounded seems fundimental without invoking a trusted third party to pool funds.
19:10:09CoinCadence:Could serial multi-sig replace a trusted 3rd party?
19:10:28CoinCadence:trustless is a big part of P2Pool ;)
19:10:53gmaxwell:* gmaxwell has mined since roughly the end of 2010, save for ~three months between moving to CA and avalons showing up.
19:11:30kanzure:i wonder whether more-informative responses from validating nodes regarding validation errors would be useful for taking account of network status for node operators. hrm.
19:11:41gmaxwell:CoinCadence: I disagree that trustlessness is actually a big part of p2pool in practice. A significant franction of p2pool users have used third party servers; which are completely trusted.
19:12:35CoinCadence:gmaxwell: true, but with the exception of NastPools new implementation they are still paid from coinbase...
19:13:09CoinCadence:our public node is open to all, 0 fee, miner username = payout address
19:13:26gmaxwell:CoinCadence: that doesn't make it trustless, thats trustlessness theater.
19:13:40petertodd:"trustlessness theater" <- nice
19:13:47gmaxwell:(and plenty of other ones have done traditional aggregated pooling on top of it)
19:13:56CoinCadence:I guess to some degree it is, but we allways encourage to set up own node...
19:14:05petertodd:though I think "decentralization theater" has a better ring to it
19:14:09gmaxwell:(To be fair, it's not _totally_ trustlessness theater, but considering the high share variance its darn near so)
19:14:47CoinCadence:would not be hard to skim shares, if your on someone eleses node you have to trust them, true
19:15:31gmaxwell:it would be perfectly reasonable for people to instead of mining to their own address, mine to a multisig address that has agreed to pool and pay out for them, then they could get very low variance, same as any other pool.. with exchange of some risk.
19:17:11CoinCadence:like it, how do you determine what each miner on a given multi-sig has contributed?
19:17:31kanzure:mining to a multisig address would not make that trustless ;)
19:17:49Adlai:but it's decentralizing the trustlessness!
19:18:03kanzure:or rather, i mean login with a multisig address or whatever
19:18:03CoinCadence:would allow for groups of smaller miners to join, and yes would require trust within the group...
19:18:36justanotheruser:yeah, pay-to-proportional-tx-output-set-hash
19:18:54gmaxwell:CoinCadence: they send the multisigners evidence of their mining seperately.
19:19:56gmaxwell:Imagine that the subpool is PPS for a moment. You mine to 3subpool and then send any (say, diff 256) shares you find to the multisigners, and then they credit you diff 256 worth of bitcoin, which you can withdraw at any time from them.
19:21:32CoinCadence:gmaxwell: I get it, the caviat is how the multisigners come to consensus
19:22:43gmaxwell:CoinCadence: thats up to them, I gave a PPS example because it simplified out that part. Without it then everyone would submit shares to them and they come to consensus among themselves about which shares are to be paid.
19:23:23gmaxwell:Consensus with a fixed set of participants is a solved problem many times over.
19:26:40CoinCadence:I wonder if we could modify the share chain to include a miner ID along with the multisig address, and use the share chain to track valid shares submitted by each miner in the cluster, then you could set a time in the future to pay based on a threshold you are comfortable with?
19:27:55gmaxwell:there isn't any modification required, you cna have arbtitary extra data, but you're missing the point there. If you only use the data in the p2pool sharechain you get the same variance as if you were paying the users directly.
19:28:20gmaxwell:(unless you allow the sharechain data rate to become large, and potentially unreasonable.)
19:30:07CoinCadence:That was another agle prposed by some of the community, make p2pool more efficient (maybe by re-writing in C++) and increase the number of shares in the chain, either by reducing expected time to share, or by extending the valid chain (or a combination of both)
19:32:07CoinCadence:*angle proposed
19:37:38gmaxwell:CoinCadence: people already were bitching about p2pool's bandwidth usage back when it had a 10 second sharechain. I think it's unlikely people would find more than double the current level acceptable.
19:38:56CoinCadence:gmaxwell: was not around then, but bandwith usage for the p2pool node is actually quite low right now...
19:39:34gmaxwell:well the share time was changed to 30 seconds which cut the bandwidth usage by a factor of 3.
19:39:53gmaxwell:I think it was pretty low at 10s too, but obivously opinions differ. :)
19:40:14CoinCadence:My node right now: 24.9kB/s
19:40:38CoinCadence:and I'm one of the mostpopular on the pool ;)
19:41:13CoinCadence:however, some hardware def. had a problem with the high restart rate
19:41:28gmaxwell:your size doesn't change the bandwidth all that much, but yea, it was about 110kB/s previously.
19:42:24gmaxwell:and people did complain about that. (consider a significant portion of the US only has DSL with 1-3 mbit/sec upstream)
19:44:18dgenr8:petertodd: any data gathered today on fee elasticity wrt a restrictive size limite will not be useful when it's needed .... in a dozen years minimum
19:46:32CoinCadence:gmaxwell: thx for your input, I'll continue to think on it...
20:30:20gmaxwell:Last call on any blockers for assigning a BIP number to this simple spec specifying lexagraphic ordering by default for p2sh multisignature: https://github.com/afk11/bips/blob/b924a75f66580ffa12b6f72b625ec1575d002691/bip-0090.mediawiki
20:31:41gmaxwell:(i.e. are there any important applications or optimizations which are broken by using this particular ordering rather than some other one which need to be discussed; the proposal is really short and can be skimmed in just a minute)
20:32:13kanzure:i looked over it and didn't see any problems, but the absence of mumble mumble does not prove mumble mumble
20:32:50gmaxwell:yea sure. the best we can do is make sure a bunch of people have seen it; this isn't cosmic importance, so I'm not looking for proof that no mumble mumble blazerzot.
20:34:51gmaxwell:This scheme cannot work for more complex scripts that use checkmultisig multiple times and reuse some of the keys (because then positions matter); but I think thats compfortably out of scope.
20:36:42afk11:i'm here if anything needs answering.
20:57:11mike420:good price
20:59:48mike420:mike420 has left #bitcoin-wizards
21:07:08arubi_:arubi_ is now known as arubi
21:07:18Eclipse:BRUH BRUH
21:07:22Eclipse:Eclipse has left #bitcoin-wizards
21:07:38bramc:petertodd, I'm planning on going to your talk at bitcoin-dev
21:08:19bramc:So, I've been working on this problem of combining multiple proofs of storage to make it so that a pool can't get a side advantage, and have come up with some fairly 'well duh' observations
21:10:07bramc:If you look at just a single generation, if you're combining multiple proofs of storage there's a hard limit to how unlikely you can make the attacker to succeed, which is that they might win in every single one of the sub-proofs
21:10:40bramc:So if an attacker has, say, 1/10 of the total capacity, and three proofs of storage are combined, then they have at least a 1/1000 chance of succeeding
21:11:50bramc:There's another monkey wrench in the whole thing, which is that some fraction of the time when they succeed as a fork it isn't really a fork - it's the main chain. It's a bit hard to evaluate that one.
21:14:14bramc:So I'm messing with details to try to get as close to that hard limit as possible. And I'm thinking the number to combine, as they explain in the holy hand grenade skit, is three, because that makes it so that a pool of size 1/10 is hardly getting any benefit, and a pool of size 1/4 is going to have a measurable advantage even with four
21:28:17kanzure:gmaxwell: perhaps it would be better to specify the exact script that it is compatible for
21:28:25kanzure:cc afk11
21:33:54instagibbs:kanzure: +1
21:53:56brisque:justanotheruser: bitnodes does some basic checks and blacklists some Sybil nodes, but it's obviously not able to be perfect.
21:56:55brisque:Petertodd: I'm not sure I understand. if you boil down Bitcoin to checkpoints than you might as well just call it an issued currency by whoever has the key.
21:57:30brisque:petertodd: in the case of ethereum, they've managed to make a GPU resistant mining algorithm that runs primarily on GPUs.
21:59:00justanotheruser:brisque: Not sure what kanzure was going for, but the best you can do is something like that, or on a small scale try to connect to many nodes with 2 connections and see if they actually validate and relay
22:05:40brisque:why with two?
22:06:51brisque:more than one doesn't tell you anything more than relaying something unique to them and seeing if it ends up on another peer. also doesn't mean they're a useful node, either. there public software now which will make a Sybil node that passes this tea.
22:08:35justanotheruser:brisque: making a block is expensive. You want to be able to tell whether the person you relayed to is relaying before the block makes its way back to you
22:09:43justanotheruser:But at this point I am completely speculating on what he meant and it may be completely different
22:23:30kanzure:we really need a name for the thing that checkpoints were that other people don't mean when they say checkpoints
22:24:00justanotheruser:centralized blockchain block ivalidation?
22:24:32gmaxwell:It's always easier to change the words you use than the words other people use.
22:26:14justanotheruser:or just get rid of them
22:26:24kanzure:i'm okay with that, am still interested in name proposals either way :P
22:26:39kanzure:(for the one that was implemented)
22:27:07justanotheruser:SNCMC - static not-consensus-mechanism checkpoints
22:27:41Adlai:consensus snapshots?
22:27:47justanotheruser:darkcoiners have also redefined "coinjoin" so they can say "coinjoin is centralized, we don't use coinjoin"
22:27:52petertodd:"trusted snapshots"
22:28:28petertodd:justanotheruser: darkcoiners can go die in a fire for that
22:28:48gmaxwell:trusted is not a great word because you can't tell if it means trustworthy or blindly-relied-on
22:29:11belcher:how the hell is coinjoin centralized?
22:29:26petertodd:brisque: thanks! we should talk there
22:29:26Adlai:through bad implementations such as 'shared send'
22:29:53petertodd:gmaxwell: trusted has this wonderful definition in many circles as "something that can fuck you over"...
22:30:16gmaxwell:petertodd: yes but in others as trust-worthy. :(
22:30:47petertodd:brisque: there's a lot of subtleties to different types of checkpoints in terms of how much auditing happens prior to the trusted checkpoint being accepted
22:30:51petertodd:gmaxwell: "blind checkpoints"
22:31:00gmaxwell:Adlai: yea, nevermind that basically everyone who has worked on or researched coinjoin implementations have told them that their service is busted. alas.
22:31:00petertodd:gmaxwell: er, I mean "blind snapshots"
22:31:12gmaxwell:petertodd: I like that better.
22:31:35kanzure:i forget whether we are trolling each other or which checkpoint definition we are talking about. lemme go look at my notes..
22:31:44petertodd:kanzure: why not both?
22:32:58kanzure:what happened to the "anti-dos checkpoint" name suggestion?
22:33:12gmaxwell:At the moment, -- (my opinion is open to change)-- I'm really not fond of any of these schemes, I'd rather just have difficulty threshold style checks to prevent DOS attacks; "consensus advice" that warns you if you're in a weird state; and tools like invalidateblock being available to users.
22:33:53justanotheruser:belcher: implementations of it are
22:34:09kanzure:initial-sync-checking-stints... i don't know.
22:34:22justanotheruser:just like how file sharing is centralized and bittorrent isn't file sharing
22:34:34petertodd:gmaxwell: I really, really, really dislike difficulty threshold, because that fundementally gives control of what is considered valid to anyone with sufficient hashing power - it's just a bad model with much worse implications than "yes, we claim this blockheader is from a valid chain"
22:34:49gmaxwell:justanotheruser: nothing about _coinjoin_ is centeralized.
22:35:04petertodd:gmaxwell: for instance, consider someone running old software in that case where the diff. threshold represents something relatively easy to attack
22:35:13justanotheruser:gmaxwell: I know, hence the bittorrent example
22:35:44justanotheruser:I am just explaining why people mistake coinjoin as being something centralized
22:35:50Dr-G2:Dr-G2 is now known as Dr-G
22:36:14gmaxwell:petertodd: such a party ultimately already has control... they can cheaply reorgnize the tip as much as they like, double spend people, claw back subsidy and fees, etc... In the case of a large reorg, if you just have all wallets go into a safe mode after it, people can just hit a few buttons and slay off any rewrite.
22:36:49gmaxwell:petertodd: and you've avoided granting tacit approval to things like these centeralized blocksigning schemes or made there be some centerally administred list of trusted values people have to update all the time.
22:36:57petertodd:gmaxwell: that's the thing, no they don't. again, remember my example of someone running some old software where the threshold is out of date
22:37:39petertodd:gmaxwell: after all, I'm only talking about the case where software ships with some blockhashes to skip validation - if the longest chain doesn't appear to include those hashes, redo validation
22:37:51petertodd:gmaxwell: I'm *not* talking about "checkpoints" as we know them
22:38:20gmaxwell:oh you talking about skipping some validation along a single particular path but not changing the consensus.
22:38:29gmaxwell:"blind shortcuts"
22:38:51gmaxwell:a "blindly trusted shortcut"
22:39:34petertodd:diff. thresholds for anti-dos are perfectly fine
22:40:20gmaxwell:it somewhat seems sad to have all this discussion to save what is only a couple CPU hours after all optimizations are in place.
22:41:11kanzure:"yes, i only have discussions that can save at least 200 million hours of cpu time globally, anything else is boring" words to live by
22:41:42kanzure:(although something i happen to agree with in principle)
22:41:56gmaxwell:The irony is that the initial shortcutting was put in bitcoin core because of an outright bug. We should have taken it back out when the real bug was fixed. (accidentally we were using the SecureAllocator for basically everything, so every allocation was mlocking and on free munlocking and bzeroing... which as a MASSIVE slowdown for verification; esp on operteron where TLB flushes were super exp
22:43:42gmaxwell:(the nature of the bug made it not show up in profiles... as much of the time was in the kernel)
22:44:27petertodd:well, that's one of the problems with having an architecture and devel philosophy where small changes like that matter...
23:14:56bramc:gmaxwell, Is there a reason to use SecureAllocator in a full node? Obviously it makes sense in a wallet, but I don't see what's secret in a node.
23:15:27sipa:bramc: only for wallet stuff
23:16:00bramc:I would argue that wallets should be completely separate pieces of software, for basically that reason
23:16:12bramc:If you want a wallet + full node you can run a wallet and a full node.
23:16:21brisque:Adlai: Shared Coin doesn't in a single way resemble coinjoin.
23:16:42brisque:here's a pig.
23:16:51brisque:* brisque holds up a porcupine
23:17:02brisque:(they'll never notice because they're not experts in pigs)
23:18:40gmaxwell:bramc: its only wallet code, just some minor mistake in the codebase caused it to get used where it shouldn't be.
23:19:12gmaxwell:"and the evidence is so very clear and present that anyone justifying software design choices by claiming that "the network is reliable" without irony should probably not be trusted to build any computing systems at all" http://queue.acm.org/detail.cfm?id=2745385
23:20:14bramc:Sound travels about a foot per millisecond, FYI
23:20:28brisque:* brisque slaps bramc with a metric ruler
23:21:12bramc:So with an old-school radio setup if you were sitting in the bleachers and listening on the radio you would hear the batter make contact with the ball on the radio at the same time as you saw it but would hear it through the direct air at a noticeable delay
23:22:24sipa:bramc: nobofy disagrees that the wallet code shouldn't be separate :)
23:22:32sipa:just hasn't been done yet
23:23:16brisque:sipa: least that isn't a comment about we have to remove the marketplace *and* the wallet from bitcoin core :)
23:23:37bramc:What do you mean by the marketplace?
23:23:58brisque:very very very early bitcoin core had a marketplace built into it, or the framework of one anyway.
23:23:59sipa:bitcoin core 0.1.5 had some unfinished marketplace built in
23:24:16sipa:no actual logic, but part of the network protcol andngui
23:26:37hearn:it was intended to be a market where blocks mined == reputation
23:26:55sipa:i never knew that
23:27:33bramc:Because we all know what reputable upstanding people professional miners are
23:28:53hearn:remember satoshi imagined that most users would mine on their home cpus
23:28:54brisque:keep in mind the network was fairly different to how it is now. you would connect to someone's IP address directly and exchange pubic keys in network messages.
23:28:57hearn:it probably made sense at the time
23:29:38bramc:There are lots of things which made sense to try at the time which are clearly bad ideas now.
23:29:39hearn:now people are trying to bring it back as openbazaar, except about a million times more complicated
23:29:56hearn:well satoshi said he realised it was a bad idea and switched to doing the rpc api
23:30:10hearn:i.e. he realised the community was growing and his time was better spent empowering other devs
23:30:56sipa:i see
23:31:30bramc:Miners can throw whatever info they want into blocks
23:31:40brisque:hearn: started today it would be released as BazaarCoin, have an IPO, and be built into a coloured ethereum sidechain.
23:32:20hearn:yeah. seems a shame.
23:32:35hearn:there's a certain elegance to the "just power through it in win32 C++" approach that satoshi used :)
23:32:53sipa:it would also run on the cloud
23:32:57sipa:mssed opportunity there
23:33:11jgarzik:hearn, rofl
23:33:22bramc:C++ is a strange choice for the bitcoin codebase
23:33:39sipa:i woukd not use anything else if i started today :)
23:33:41jgarzik:I thought so at first -- but in hindsight it's better than the other choices
23:33:46hearn:sipa: "it's decentralised!! all you have to do is install this precise build of ubuntu node mongodb bitcoind go + twelve other libraries and then register for an AWS account!"
23:34:09sipa:i woukd like C more, because of easier control over resources
23:34:31sipa:but needing to deal with manual memory management is such a pain it's not worth it imho
23:34:37bramc:Java and Python are both much less worrisome from a security standpoint, and of course the actual mining should be in C extensions anyway.
23:34:40sipa:consensus code i would write in C
23:34:46hearn:most code doesn't require super explicit control over memory
23:35:12jgarzik:smart C++ engineers out several problematic patterns programmers keep resurrecting. I tend to think C++ is superior to C even for consensus -- and this is coming from a C fan who wrote a C compiler & kernel C code for decades...
23:35:28jgarzik:reluctant conclusion
23:35:41hearn:heh. i'd do it all in java or kotlin :) except for the core ecdsa. that, it seems, pretty much has to be in C or assembly
23:35:45bramc:Does the SecureAllocator using a kernel memory call for each allocation or does it do one big pool via a secure call and use that until it needs another one?
23:35:52sipa:if i could just have structs with constructors and destructors in C, i would use it
23:36:01sipa:bramc: the most naive way possible
23:36:01gmaxwell:bramc: it's not as strange as you might guess.
23:36:16hearn:sipa: well you can write C++ that way
23:36:22hearn:sipa: but do you really want to write your own linked lists ...
23:36:27bramc:I'm told that C++11 is much less problematic than the older versions
23:36:44sipa:hearn: i have written linked list implememtatiins in C several times :)
23:36:49sipa:hearn: and hash tables
23:37:01sipa:and ni, i don't want to do that again
23:37:04hearn:exactly :)
23:37:22gmaxwell:jgarzik: in general consensu code doesn't do any of the things that are espeically footgunny there, and C++ also introduces a lot of oppturnity for shocking behavior that even surprises the foremost C++ experts that you can find; which is a pretty big liability when writing anything whos behavior must be absolutely and totally unambigious.
23:37:24sipa:but to the extent possible, i think consensus code should have as little allocations at runtime at all
23:37:26bramc:gmaxwell, How is it not so strange? What are th ebenefits?
23:37:55hearn:sipa: why? it's not hard real time.
23:38:08sipa:hearn: being able to reason about memory usage
23:38:25sipa:which does not require no-allocations at all
23:38:35sipa:but minimizing them simplifies things
23:38:50sipa:and c++ really makes it easy to glace over really inefficient things
23:39:01sipa:but maybe that's just a sign of a bad programmer :)
23:39:04gmaxwell:bramc: well consider the original version of bitcoin it was only about 10kloc with a GUI. We have never had a memory safty bug hit us in production-- whole classes of errors that are easy to have in C are much less likely with relatively modern C++. It's not good, however, at avoiding hidden behavior...
23:40:14brisque:gmaxwell: all the more hell for reimplementations I suppose.
23:40:29hearn:i think we got incredibly lucky with satoshi's code, especially given the total absence of any modern testing methodology
23:40:34gmaxwell:bramc: e.g. the only memory leaky ish bugs we've had IIRC in bitcoin core were just container datastructures which had no upper bound, no actual memory leaks in the sense that you get them in C that I can recall.
23:40:48gmaxwell:hearn: I think you're incorrect about testing methodology, it just wasn't made available to you.
23:41:01jgarzik:gmaxwell, and a lot fewer memory corruption type bugs/errors on average
23:41:04hearn:that's what i mean by reasonable - satoshi tested by writing little standalone apps then throwing them away
23:41:16hearn:sorry s/reasonable/modern/
23:41:52jgarzik:sipa, ideally consensus code should do zero allocations... embedded situations want you to pass in things like heap & stack
23:41:56bramc:My coworkers tell me that in C++ 11 you mostly use smart (i.e. ref counted) pointers and unique pointers, so you basically don't do any memory management. Python winds up being basically the same thing
23:42:00hearn:everything about bitcoin 0.1 screamed "this is going to be riddled with exploitable bugs" and it wasn't
23:42:27hearn:bramc: refcounting *is* memory management, but yes it can avoid issues like double frees when done properly and consistently so it's better than fully manual if you can take the code size/perf hit.
23:42:36hearn:(bitcoin doesn't do refcounting)
23:43:06bramc:A lot of the exploitability of a piece of software has to do with how good the developers are. Some people are just plain sloppy
23:43:31bramc:But I prefer to do development in Python for purely development time reasons.
23:43:43gmaxwell:There is very little of satoshi's code left in Bitcoin core. 0.10 replaced much of what had been remaining.
23:44:06bramc:hearn technically Python is garbage collected as well but it turns out in practice most of the work is done by ref counting. I was simplifying.
23:45:16gmaxwell:in bitcoin there are very few long lived objects, and basically none without simple linear ownership, memory management isn't generally a huge issue.
23:45:43sipa:bramc: if you wrote consensus code in python, i'm sure you'd get forks when new python versions were introduced :)
23:46:17sipa:jgarzik: yup, libsecp256k1 only does allocations at init time
23:46:18bramc:sipa, No Python is extremely stable in terms of basic syntax.
23:46:34sipa:bramc: well, let me know if you :)
23:47:04hearn:gmaxwell: my fear is that as the years go by and multi-threading gets more common ownership will get more complex. we already have locking issues with the RPC threads vs main threads, etc. threading is a fertile source of double free and ownership type bugs.
23:47:14bramc:What does the bitcoin codebase use for strings? I'm told null terminated strings have mostly gone the way of the dodo.
23:47:17hearn:but yes it's much less of an issue than say, webkit
23:47:38gmaxwell:bramc: uh. I've had simple thing like adding an import in python change the behavior of unrelated code.. because the import changed things about the interperter enviroment.
23:48:18bramc:gmaxwell, I'd be curious for details on that one. I've never had anything like that happen to me. I also don't use any crazy tricks in Python.
23:48:42bramc:(Well, there's utah data center, but that's just for testing purposes)
23:50:06bramc:If you make a codebase monothreaded and use a consistent and reasonably safe string class that fixes most of your security problems regardless of which language you use.
23:50:12hearn:bramc: std::string
23:50:24hearn:bramc: yeah null terminated strings are a C thing only these days
23:51:13brisque:hearn: and PHP :P
23:51:14bramc:I'm old enough to have gotten serious criticism as a know it all kid for shitting on null terminated string as being obviously idiotic when I first started out.
23:51:19gmaxwell:bramc: god knows what strings are being used for at all. Every time someone has added some string thing to the Bitcoin system (e.g. not in some UI facing role) they've managed to add some kind of vulnerability or another, strings have no real place in this system.
23:55:11jgarzik:most pragmatic implementations store string length & null terminate for safety (+ speed of rendering a C-string upon request)
23:55:41sipa:which is what std::string does actually
23:56:23bramc:python3's biggest difference with python2 is the separation of string into separate binary and unicode string data types. It seems utterly barbaric for them to be crammed into the same structure once you're used to that.
23:57:05bramc:I'd like to JCF-terminate my length-delimited strings for testing purposes :-P
23:58:14bramc:gmaxwell, Well it implicitly must use binary strings all over the place, at least when it's doing network traffic
23:58:21hearn:we tend to just call those byte arrays
23:58:37hearn:though using std::string to hold arbitrary binary data is not uncommon in many c++ code bases
23:59:58justanotheruser:hearn: what should be used? a vector of bytes?