--- Log opened Wed Jul 17 00:00:57 2013 00:01 < petertodd> (remember that if this is a timestamping facility any node wanting to know the current time simply gets a nonce timestamped, and then they know what time it is!) 00:11 < Luke-Jr> I don't see how nLockTime can discourage forward-dating blocks 00:11 < Luke-Jr> and there is no 2hr window backward.. 00:12 < Luke-Jr> well, I guess if miners are behaving there is <.< 00:19 < petertodd> The problem is a block being created with nTime > actual time, and the incentive is to get a head start on other miners to put, say, a high-fee nLockTime in the block you are creating. 00:21 < Luke-Jr> petertodd: but nLockTime only sets a minimum time, it cannot set a maximum 00:22 < petertodd> but that's it, if I have a 1BTC fee tx, with nLockTime expiring in two hours, why not take the increased orphan chance and set nTime on my block to two hours ahead/ 00:22 < petertodd> ? 00:22 < petertodd> yet if we allow that incentive, it's very bad for consensus 00:23 < gmaxwell> ha. We can fix. 00:23 < gmaxwell> it's a soft forking fix. 00:23 < gmaxwell> use the last blocks ntime, not this one. 00:23 < Luke-Jr> is sipa's secp256k1 branch reasonably stable? 00:23 < petertodd> gmaxwell: that's what I said... 00:24 < gmaxwell> petertodd: sorry I just read the last couple lines. 00:24 < Luke-Jr> petertodd: AFAIK we already don't relay transactions with time in the future? 00:24 < gmaxwell> petertodd: well I agree. (or not even the last block— it could use the minimum time) 00:24 < petertodd> gmaxwell: The problem is, that's only a fix if mining power is well distributed, it actually makes things worse because if there is a lot of profit to be gained the miners with a lot of hashing power still have the incentive, and it's to a much greater degree. (their orphan rate is less) 00:24 < Luke-Jr> gmaxwell: the minimum time will be earlier than the last block's :p 00:25 < gmaxwell> Luke-Jr: sure, but that doesn't change it really. Presumably if people start locking in the future miners will run nodes that take what they get and selfishly horde them, creating incentives for all miners to run good collection networks. 00:25 < petertodd> Luke-Jr: sure, but there are lots of ways to learn that a tx exists 00:26 < gmaxwell> petertodd: one of the reasons that the min is important there is because (1) it's hard to advance, and (2) when you advance it you raise the difficulty. 00:26 < petertodd> gmaxwell: I was working on figuring out the expected return - the math is really ugly 00:27 < gmaxwell> petertodd: a worst case expected return may be easier. 00:27 < petertodd> gmaxwell: Worst case is easy - your block is orphaned. 00:28 < petertodd> gmaxwell: See the issue is that once I find a block, the other side needs to find two blocks to beat me. As time goes on more of the other sides hashing power will accept my from the future block as valid, so then you get the next level where the remainder needs three blocks and so on. 00:28 < petertodd> gmaxwell: Pretty sure it can't be done as a closed-form equation. 00:30 < petertodd> gmaxwell: I don't think minimum time works either, because you still get to manipulate it by creating blocks in the future, although the ability too is definitely less. If I could show you'd need >50% hashing power to do anything interesting I'd be set. 00:31 < Luke-Jr> petertodd: hmm, is block-uneconomic-utxo-creation basically just an older revision of what Gavin did in 0.8.2? 00:31 < gmaxwell> petertodd: moving the minimum time forward needs the coperation of >50% of the hashpower over the small median window. 00:32 < petertodd> Luke-Jr: It's what Gavin did but non-hardcoded. I'd emphasize the better, not the older. :P 00:32 < Luke-Jr> petertodd: will you be rebasing it despite its closed status? 00:32 < Luke-Jr> actually, what about Gavin's is hardcoded? <.< 00:33 < petertodd> gmaxwell: Yeah, but you have to assume a steady stream of these incentives. 00:33 < gmaxwell> petertodd: right, so you have some force that turns all miners into a conspiracy. 00:34 < petertodd> gmaxwell: exactly 00:34 < petertodd> gmaxwell: nLockTime by time should have never been added in the first place, but it's such a nice idea on the face of it 00:35 < Luke-Jr> softfork so nLockTime requires data on what block a transaction was created at, and enforces the 10 min per block <.< 00:36 < petertodd> Luke-Jr: ? 00:36 < Luke-Jr> petertodd: for example, if you nLockTime for 1 day from now, it also enforces 144 blocks passing too 00:37 < Luke-Jr> so block count must be >now+144 AND time must be >now+24h 00:37 < Luke-Jr> not perfect, but might help 00:37 < petertodd> Still doesn't help in the usual case where mean interval is < 10 minutes, because you're back to only caring about time. 00:38 < Luke-Jr> usual now, but not eventually 00:38 < petertodd> Right, you've solved half the problem, when measured over the entire lifespan of Bitcoin, and only approximately half. :P 00:39 < Luke-Jr> theory is so much nicer than practice <.< 00:39 < gmaxwell> I'm forgetting why this is a problem again? If miners mine blocks early, people will just artifically inflate their times or switch to height locking. 00:39 < petertodd> The problem is you're incentivising miners to make the 2hr window for block acceptance effectively shorter. 00:39 < petertodd> Thus requiring accurate clocks for consensus. 00:39 < gmaxwell> if miners do this consistently they'll drive difficulty up too which wouldn't be in their interest. 00:39 < Luke-Jr> ^ 00:40 < petertodd> gmaxwell: It's only a fixed 2hr offset, that just drives difficulty up by 0.5% 00:40 < Luke-Jr> and on top of that, you'd just end up treating nTime with a minus-2-hours :p 00:41 < Luke-Jr> if everyone does it, it's predictable. 00:41 < petertodd> More to the point for any individual miner the marginal difference if they do it is effectively zero. 00:41 < gmaxwell> consider, why why cant the 2 hour window be 24 hours? 00:41 < petertodd> Luke-Jr: But that's the problem, if everyone does it, and people respond, then you can extend the interval even further! 00:41 < Luke-Jr> petertodd: how? 00:41 < petertodd> gmaxwell: It should have been more like 24 hours in the first place... 00:42 < Luke-Jr> you don't change the 2h rule 00:42 < Luke-Jr> you just assume miner times will always be up against it 00:42 < gmaxwell> Luke-Jr: move your clock/window forward so you dont reject stupid blocks. 00:42 < petertodd> Luke-Jr: Again, the issue is the effect on *consusus*. I don't care when the tx gets mined, I care that miners are incentivised to break consunsus for anyone without NTP. 00:43 < petertodd> The problem is no matter *what* the window is, there is an incentive to mine as close to the window as possible to accept a tx sooner than your competitors. 00:43 < petertodd> It could be a week and people would still have an incentive to set nTime + 1 week - 1 second 00:44 < Luke-Jr> if nTime is future, wait until that time before relaying it? <.< 00:44 < gmaxwell> and once people did that, you'd want to start accepting blocks that where nTime + 1 week because god knows you don't want to reject a block if your clock was 2 seconds slow and most hashpower accepted it. 00:44 < petertodd> About the only thing that might change that is if the rule was nLockTime > nTime of last block, and then after that being allowed to include a tx was based on H(txhash, last hash) or similar 00:45 < petertodd> gmaxwell: exactly, the fundemental issue is there is no good incentive to set nTime accurately other than miners rejecting your blocks, and nLockTime sabotages that 00:45 < petertodd> gmaxwell: (timestamping could do, but the cause->effect is less obvious) 00:45 < Luke-Jr> I guess I just incentivized always setting nTime to the minimum then 00:45 < Luke-Jr> [04:32:26] petertodd: will you be rebasing it despite its closed status? (block-uneconomic-utxo-creation) 00:46 < petertodd> Luke-Jr: again, relaying does nothing - consider the case of nLockTime'd fidelity bonds where it's guaranteed 100% of the hashing power know (why I wrote the spec as by-block-height in the first place) 00:46 < petertodd> Luke-Jr: sure 00:46 < Luke-Jr> petertodd: I mean delaying relaying the BLOCK 00:46 < Luke-Jr> ie, increasing the risk of it being stale 00:47 < petertodd> Luke-Jr: then you have your mining pool connect directly to other mining pools playing the same game 00:47 < petertodd> you have to assume perfect information knowledge in this stuff, at least if you're writing worst-case academic papers 00:48 < gmaxwell> petertodd: so ... prior block vs minimum time. 00:48 < petertodd> see, that's why I was talking about timestamping, because it provides a way for all users to set their clocks to what the majority of hashing power thinks nTime is, sidestepping the problem 00:48 < gmaxwell> petertodd: what are your arguments there? 00:48 < petertodd> gmaxwell: minimum time is definitely stronger because it involves more hashing power 00:49 < petertodd> gmaxwell: users would prefer minimum time - easier to understand why the tx isn't getting mined 00:49 < gmaxwell> sidestepping the problem < that doesn't sidestep the problem, it would allow the majority of hashpower to mine difficulty down to 1; also moots nlocktime as _time_ being more reliable than a height. 00:49 < gmaxwell> petertodd: plus, you can just add a constant offset to your nlocktime to adjust for the expected minimum lag. 00:51 < petertodd> gmaxwell: yes, it creates a new problem, but it did sidestep the existing one :P 00:51 < gmaxwell> petertodd: yea, lol, creates an inflation attack. Keep it up and you'll be qualified to create an altcoin. :P 00:52 < gmaxwell> (sorry, hah, I'm not poking fun at you, I'm poking fun at all the altcoins that "solved the Foo problem" where foo is something no one else thinks is a problem and they totally broke security as a side effect) 00:52 < petertodd> gmaxwell: yup, now you see how it only sidesteps the problem truly when there is enough hashing power setting their clocks back, IE 50% honest, which is better 00:53 < petertodd> gmaxwell: without the timestamping, nodes have the consensus failures, which can be attacked, likely it trades off one risk for a more existential risk 00:53 < petertodd> gmaxwell: and it's a good excuse for timestamping, lol 00:54 < gmaxwell> I thin the min solves the consensus failure so long as hashpower is well distributed. 00:54 < petertodd> yeah, I'm thinking min is probably the best we can do 00:55 < petertodd> other than disabling nLockTime of course 00:55 < gmaxwell> only time based, height is still safe. 00:55 < petertodd> it'd be good if miners had a "fuzzy" window too, so if they get a block really close to the 2hr window, they'll delibrately try to orphan it, but such stuff can't be more than "it's in the codebase so hopefully people will be slightly economically irrational" 00:56 < petertodd> gmaxwell: of course 00:56 < petertodd> nLockHeight is ok :P 00:56 < gmaxwell> doesn't even have to be in the codebase, natural clock skew accomplishes that. 00:57 * Luke-Jr decides it makes sense to increase the accepted timespan on blocks for Eligius… 00:57 < Luke-Jr> :p jk 00:57 < petertodd> hopefully... I suspect pools, and miners in general, are much more likely to be running Linux and thus will have ntp enabled 00:57 < gmaxwell> their time still gets randomized due to the goofy medianing stuff. 00:57 < petertodd> measured skew on my nodes, IE ntp vs GetAdjustedTime() is almost always under a second or two 00:57 < Luke-Jr> doesn't Windows enable NTP by default now? 00:57 < gmaxwell> which, btw we've _still_ never closed that vulnerability. 00:58 < petertodd> gmaxwell: Yeah, all this stuff gets even more interesting if you sybil even part of the network... 00:58 < gmaxwell> petertodd: GetAdjustedTime has been bad in the past but seems to be okay since I reset it like.. two years ago now. 00:59 < gmaxwell> I wonder how it got goofy in the first place... but stays pretty good since it was initially fixed. 00:59 < petertodd> In fact, adding pow-timestamping to the GetAdjustedTime measurement could have value. 00:59 < petertodd> gmaxwell: oh, so previously you were seeing much bigger skews? 01:00 < gmaxwell> petertodd: the vulnerablity I was speaking of was the the maximum GetAdjustedTime skew could get a node and a miner producing blocks that the other will reject. 01:00 < gmaxwell> petertodd: yea, IIRC > 30 seconds. 01:00 < gmaxwell> Then I sybled the network and reset it. 01:00 < gmaxwell> and it seems to have stayed reset. 01:00 < petertodd> wtf 01:01 < gmaxwell> may have just been some initial symetry breaking when the network formed that made it get stuck offset. 01:01 < petertodd> Maybe we should randomize the GetAdjustedTime() calculation a bit to try to make sure symmetry is broken again naturally? 01:01 < petertodd> ...though I'd rather thoroughly understand how that happened first... 01:02 < gmaxwell> I mean, it's just applying a median operation to the peers median operations. So if there ever was a wrong majority it would take over the network and then stick. 01:02 < petertodd> Makes sense 01:03 < gmaxwell> e.g. if you had a network of three nodes, then you add 4 more who all had times +30 seconds off.. the three would also jump to +30 and then so long as you never introduced a majority all at once again it would stick that way. 01:03 < petertodd> Then don't give our peers out adjustedtime, give them our localtime. 01:03 < gmaxwell> giving adjtime is good for consensus. 01:03 < gmaxwell> but not accuracy. :) 01:04 < gmaxwell> I suppose it could give adjtime + λ*error for some small λ. 01:04 < petertodd> Yeah, and calculate what's needed to break the consensus. 01:04 < petertodd> semi-break it 01:04 < petertodd> Heh, my patch to add adjustedtime to getinfo was probably more useful than I thought... 01:05 < gmaxwell> e.g. if adjtime is higher than local, give adjtime-1. if the whole network gets ahead then everyone will be -1... and it will slide to that. 01:05 < petertodd> Ah, that'll work 01:07 < gmaxwell> I think ±1 is actually sufficient. it doesn't matter if it adjusts slowly... ±1 is both necessary and sufficient. 01:08 < petertodd> We should also make bitcoind use local time for block creation, not adjusted time. 01:08 < gmaxwell> I dunno about that. 01:09 < petertodd> It's a more true vote about what miners clocks are set too. 01:09 < petertodd> And you can take the min of both in cases where clocks are ahead. 01:09 < petertodd> IE, if my local clock is ahead, don't create a block that the local adjustedtime consensus would reject. 01:10 < gmaxwell> alternatively, I'd offer that it should just stop mining if its too far off. 01:10 < petertodd> That too, but people will be pissed at losing revenue... 01:10 < gmaxwell> yea, well, fix your clock. :P 01:10 < gmaxwell> too far could be ±65 minutes. 01:11 < petertodd> Using local, but sanity checking it against adjusted, has pretty much all the benifit minus the risk. (modulo adjusted < getmintime, but that can be the "stop mining" condition) 01:11 < gmaxwell> the key point is just making it so that a network attacker can't max-skew two nodes in opposite directions. 01:11 * Luke-Jr ponders if miners would be agreeable to randomizing their nTime within ±90 minutes just to discourage timestamping abuse <.< 01:12 < gmaxwell> petertodd: I would be willing to bet a minority of hashing power is run on systems with NTP setup and working. 01:12 < petertodd> Sure, but having an accurate vote for time could be useful by letting you see if such max-skew is being attempted. (for non-miners) 01:12 < gmaxwell> I would also be willing to bet that a majority of that which is using ntp can have their time reset by comproming two hosts. 01:13 < petertodd> Luke-Jr: timestamping doesn't give a damn about +-hours - bitcoin is too inaccurate for that 01:13 < gmaxwell> petertodd: a 'vote for time' is worthless unless there is a strong incentive to be honest about it. 01:13 < petertodd> Yeah, ntp compromiseis scary... 01:13 < Luke-Jr> :P 01:14 < petertodd> I'm not suggesting this to make timestamping applications more accurate, I'm simply suggesting this as a way for nodes to better know if the miner consensus is different than their local adjusted time. 01:14 < gmaxwell> petertodd: and more recent ntp software no longer does the 128 second stuff anymore, you could move chrony 10 years into the future with a majority of peers, IIRC (unless they've fixed that) 01:15 < petertodd> It's too bad the atmosphere is so thick: we could figure out local time by running memtest and analyzing the rises and fall as the earth blocks the radiation coming from the sun. 01:15 < petertodd> (then set the window to 1 week) 01:15 < gmaxwell> petertodd: plus there is huge lags on mining. You'd do better to put a ecdsa public key in blocks and have miners announce announcement timestamps for the blocks they made. 01:15 < gmaxwell> I happen to like being alive and not irradated. 01:16 < petertodd> gmaxwell: If you want to reduce the lag, just broadcast sub-target difficulties. 01:16 < Luke-Jr> lol 01:16 < petertodd> gmaxwell: Ok, how about we require bitcoin to run on computers with shittier memory then? 01:16 < gmaxwell> petertodd: I mean there is huge lag in just issuing out work to miners, scanning it, submitting results. 01:17 < petertodd> gmaxwell: That's seconds, I'm thinking tens of minutes is what matters here. 01:17 < petertodd> gmaxwell: Again, this *isn't* for timestamping data! 01:22 < Luke-Jr> gmaxwell: well, who is to say what step the timestamp is meant to be for? :P 01:23 < gmaxwell> any field is defined by its validity rule. 01:30 < petertodd> except for nonce which is defined by the easiest implementation on an ASIC... 01:32 < petertodd> between nonce, version, and timestamp you could commit to 64-bits of data directly in the blockheader, but that's ASIC incompatible 01:33 < petertodd> (efficient mining in general incompatible really) 13:43 < jgarzik> amiller, can you expand("Amiller's high hash highway stuff") 13:43 < jgarzik> ? 13:44 < amiller> jgarzik, i can try 13:45 < amiller> the scenario is, you have a large number of proof of work solutions, and you want to check that their total-work is at least W in total. 13:46 < amiller> lets say they're all at the same difficulty 13:49 < amiller> if it's inefficient to check all of them individually, then the goal is to check just a sample such that it's unlikely the sample could be made in shorter time than W. 13:51 < amiller> (i'm going slow to try not to make mistakes here) 13:53 < amiller> i'm starting to feel like i made a bunch of mistakes or at least unnecessary steps in the solution i had previously 13:53 < amiller> but the basic requirement is that you need to be able to prove that an earlier block is committed to in a previous block 13:54 < amiller> and right now the only way to do that is to traverse backwards along the chain 13:54 < amiller> but you can do this faster if you commit to a skip list (or probably even just a merkle tree to be simpler) that has pointers further back than just the previous block. 13:55 < amiller> you know how the blocklocator works right? it's supposed to make it easy to find the intersection point between two blocks, and it does so by letting you jump back an exponential number of blocks? 13:56 < amiller> basically if you commit to a structure like that in the block header then you can do a secure diff between blockchains. 13:57 < amiller> the main application of this is faster SPV bootstrapping because you can quickly/securely estimate which chain is "longer", making it harder for someone malicious to lead you on a DoS goose chase 14:04 < gmaxwell> Personally, I think reverse header fetching is better. It's not better in the asymptotic complexity case, but under the assumption that the current difficulty is only a small factor away from the sum of the far past (exponential growth), it achieves the same security with only p2p behavior changes. 14:06 < gmaxwell> (the idea there that the difficulty of the most recent blocks is enough to make creating a goose chase very expensive) --- Log closed Thu Jul 18 00:00:59 2013