--- Log opened Tue Sep 10 00:00:23 2013
01:05 < sipa> gmaxwell: only around 50% of possible x coordinates lay on the curve
01:05 < sipa> as for each x, there are 2 y coordinates
01:06 < sipa> and the field amd group sizes are very similar
01:06 < gmaxwell> yes they're similar, but the twist group order has a bunch of factors.
01:06 < gmaxwell> So it should be pretty inexpensive to solve the DLP over it (though I've never done it!)
01:06 < gmaxwell> (maybe I should try)
01:08 < gmaxwell> sipa: I guess I knew that 50% of the X were on the curve, as I wrote that to mike in an email this morning! but after staring at a bunch of math I wasn't seeing the missing condition after key recovery.
01:08 < gmaxwell> I guess I'll work it out on paper.
01:09 < phantomcircuit> gmaxwell, hehe
01:09 < phantomcircuit> i hate when that happens
01:10 < phantomcircuit> "WAIT I KNOW THIS"
01:10 < gmaxwell> it's not too hard to reason yourself into corners.
01:17 < gmaxwell> sipa: fwiw, http://en.wikipedia.org/wiki/Pohlig%E2%80%93Hellman_algorithm
01:23 < Luke-Jr> gmaxwell: do HD wallets have a possible privacy vulnerability where if you can identify N of them maybe-in-sequence, you can figure out the master pubkey?
01:25 < gmaxwell> Luke-Jr: No, not without like .. most impressive crypto break ever.
01:25 < Luke-Jr> hmm ok
01:27 < gmaxwell> the most obvious way to do this would be to crack two of their private keys in sequence, then find their difference, then search for an extended public key,i such that i and i+1 give you that difference. This is made hard because the extended public key goes through sha512 hmac.
13:55 < Guest4867> gmaxwell: re CoinJoin: if the outputs are sorted by signature, then doesn't that achieve a random shuffle?
13:55 < Guest4867> in other words the person proposing the join no longer knows the identity of outputs
13:56 < maaku> ^^ was me
13:58 < maaku> the participant contributes a blinding, and the proposer contributes the signature, but separately there's no way for either party to figure out what the unblinded signature will be, and therefore the final ordering
14:00 < gmaxwell> I think that sounds fine, but I may not understand what you're trying to solve there. e.g. just putting them in the order they were disclosed would be okay if parties waited random amounts of time to disclose
14:02 < gmaxwell> petertodd: okay, so I've got an idea for a solution to your asymetrically memory hard POW function. I'd give it an 90% chance of working, and a 50% chance of being practical.
14:04 < gmaxwell> petertodd: here is how you do it.  First define some function that consists of a fixed sequence of adds and multiplies which cannot be algebraically simplified without knowing the values in question.  E.g. I think x=a*b+c*b+c*b+c works
14:04 < gmaxwell> petertodd: now, go find a fully homorphic encryption scheme... e.g. supports both adds and multiplies.
14:05 < gmaxwell> petertodd: now when the client connects you give him a,b,c under homorphic encryption with a key known to you and tell him to run the operation chain for difficulty steps.  He memorizes results.
14:06 < gmaxwell> When he finishes he tells you the last one, and you challenge him for intermediate values however often you like.
14:06 < gmaxwell> Because you know the keys and the values of a,b,c you can compute the outcome directly, while he's forced to operate sequentially (under the slow homorphic encryption too)
15:08 < gmaxwell> maaku: Oh I just saw your coinjoin thread. I will be really sad if you aren't able to get funded to work on this.
15:10 < maaku> i would be suprised if i didn't - there's lots of deep pocketed people that want privacy for their bitcoins, but we'll see
15:11 < maaku> sometimes for nefarious reasons i'd rather not support, but i don't see how to add privacy without also enabling that
15:12 < gmaxwell> Right thats my view too. Thats just a nature of technology all of it is dual use.
15:13 < maaku> gmaxwell: what i'm trying to do is make it so that only you know which outputs are yours, without requiring a complex multi-party computation and all the requisite overhead
15:13 < gmaxwell> And if some of the flow of nefarious funds can be redirected to help out the public, then great.
15:13 < maaku> unless of course you're only joining with one person other person
15:14 < gmaxwell> maaku: right. So, some party is acting as a "hub" that e.g. merges up the signatures. Why can't they just pick the ordering at random?
15:15 < maaku> gmaxwell: that was my protocol before, and it has the disadvantage that although the mapping is obscured through blind signing, the hub can keep records
15:16 < maaku> but if the protocol is the hub manages collecting blinded tokens, signing them, and then colledting unblinded signatures, and *then* sorts by unblinded signature value
15:16 < maaku> the hub has no way of knowing any more than anyone else what the ordering is
15:17 < gmaxwell> maaku: What kind of records? I assumed that the unblinded outputs would be returned to the hub over a seperate anonymous connection.
15:18 < maaku> true
15:19 < maaku> ok all the sorting does is provide a deterministic ordering, so the hub isn't required for the last step (building the transaction)
15:19 < gmaxwell> (that requirement is kind of lame, considering that there really does not exist a good solution for those— tor is good enough for casual privacy at least)
15:19 < maaku> that means the protocol already has the feature i wanted. cool
15:19 < gmaxwell> maaku: okay good I think we're on the same page.
15:20 < gmaxwell> and yea, making the sort a function of some data known to all the particiants is fine and good and might simplify it in practice. Don't use data in the transaction to sort however, you don't want CJed transactions to be more identifyable in the blockchain than the need to be.
15:21 < maaku> yes, sort the signatures of the outputs, not the outputs themselves
15:21 < maaku> yeah i'm not happy with revelation over an anonymous connection, but it seems we're venturing into somewhat unexplored territory to avoid that...
15:22 < maaku> hopefully it's something that can be added later
15:22 < gmaxwell> well, at least I do think the multiparty computation route requires _only_ sorting. ... which isn't so bad. But I think trying to do that at the front would be suicide.
16:11 < nanotube> fwiw, i run full node on my main comp. and following that discussion spun up another node on a vps. gmaxwell wanna peer? :)
16:12 < gmaxwell> nanotube: make it accept hidden service connections too?  see doc/tor.md  :)
16:13 < nanotube> good idea.
16:14 < gmaxwell> My laptop is 5yljdotwhmx65nlk.onion  my main mining node at home is outbound only right now, but I should fix that.
16:15 < nanotube> ok. i'll get tor up in a bit.
16:16 < nanotube> is it possible/advisable to run a single node behind both tor and non-tor?
16:17 < nanotube> i see it is possible, as per doc. is it advisable? :)
16:19 < gmaxwell> If you're using tor for privacy, no. If you're using it to provide network services, yes— since you'll bridge tor-world and non-tor-world.
16:20 < gmaxwell> eventually we'll have to deal with DOS attacks coming via hidden services... (thats actually part of the motivation for the "asymetrically memory hard POW" I was nattering on above)...
16:21 < gmaxwell> so in theory the only real downside to doing both right now is that maybe a hidden service only dos attack takes out your node on both networks at once. I think thats an acceptable risk right now.
16:47 < nanotube> heh yea, i was taking a walk and realized that if i'm doing it for network health, it'd probably be good to bridge, since if everyone was tor-only it wouldn't work. come back and see you've confirmed my thinking. :)
16:51 < gmaxwell> in the future I expect that smarter networking will constrain resources so that a dos attack on one side only hurts that side.
18:23 < gmaxwell> oh god, someone some stupid reporter got the idea that the node wedging transactions were the result of someone creating bitcoin out of thin air by successfully mining a block
18:24 < gmaxwell> and it took an excruciating amount of effort to break them of the mental model where nodes simply trust everything in a block and security comes "because no one person can make a block"
18:30 < phantomcircuit> gmaxwell, lol reporters
18:30 < phantomcircuit> gmaxwell, http://pastebin.com/raw.php?i=Hjrrg3kX
18:31  * gmaxwell types /axwell   whew
18:33 < phantomcircuit> gmaxwell,
18:33 < phantomcircuit> ""Isn't anonymity one of the biggest reasons lots of people support Bitcoin?" one member of bitcointalk.org asked last month, in a comment echoed by other users. "So that the centrally controlled banks/governments don't control personal transactions or even have records of those private transactions?""
18:33 < phantomcircuit> makes me want to cry
18:35 < gmaxwell> I love it that they don't attribute there, because the guys name was problably like JohnGaltDickSlapperCyberCunt1996
18:36 < phantomcircuit> https://bitcointalk.org/index.php?topic=274186.msg2938172#msg2938172
18:36 < phantomcircuit> actually it's just Chronikka
18:36 < phantomcircuit> sooooooo
18:39 < gmaxwell> maaku: thanks for that RapidBalls thread. :P
18:39 < gmaxwell> maaku: phantomcircuit actually has wallet fixes that make the behavior not-exponential.. but I wasn't super eager to point them to them when it sounded like they were doing something spammy (lines you correctly read between even when they couldn't… :) )
18:40 < gmaxwell> ("rapidballs" totally sounds like a forum username too)
18:40 < phantomcircuit> gmaxwell, there's actually still a ton of hilariously inefficient behavior in the wallet
18:41 < phantomcircuit> but it's all acting on structures in memory
18:41 < phantomcircuit> so
18:41 < phantomcircuit> shrug
18:41 < phantomcircuit> i would however really like to break out the protocol rules into rule modules
18:41 < maaku> heh, got an unexpected 0.5btc got out of it.. which is lucky because my first inclination was to be a snarky douchebag and I almost was
18:41 < phantomcircuit> since at the moment network rules and soft node rules and anti dos rules are all mixed together
18:42 < maaku> newbie named RapidBalls spamming the network is asking for it ;)
18:43 < phantomcircuit> gmaxwell, sipa do either of you have any ideas on the structure/naming for classes that contain the network/soft/antidos rules?
18:43 < phantomcircuit> i was thinking something as simple as a class with static const methods
18:45 < gmaxwell> maaku: yea, in any case, to keep up your contracting business there I thought I'd tell you about phantom's fixes.
18:47 < phantomcircuit> lol
18:56 < midnightmagic> nanotube: which write-up was that again?
18:58 < midnightmagic> (that convinced you to spin up another node)?
18:59 < maaku> yeah thanks i didn't know about phantomcircuit's fixes
18:59 < phantomcircuit> maaku, did you manually create transactions for them or something?
19:00 < maaku> no, just told them about blocksize limits and such. they're really new to this
19:01 < maaku> he's making something like 500 transactions/minute, and wondering why bitcoin is behaving slowly
19:02 < phantomcircuit> why is he creating 500 transactions/minute...
19:02 < phantomcircuit> yeah my patches would improve his bitcoind performance but he'd just end up with a bunch of unconfirmed transactions
19:03 < gmaxwell> phantomcircuit: because his name is "RapidBalls" what do you think?
19:03 < phantomcircuit> lol
19:03 < phantomcircuit> so because he's being a dick
19:03 < phantomcircuit> got it
19:03 < gmaxwell> the username alone tells me its some derpy gambling site that hasn't figured out that they can do something other than one transaction per bet.
19:03 < phantomcircuit> or that
19:03 < gmaxwell> I promise that that their visa handling people would shut of their service faster than you can say "rapidballs" if they started running 500tx/minute.
19:05 < phantomcircuit> yeah for sure
19:06 < phantomcircuit> 500tx/minute is maybe what something like walmart does
19:06 < phantomcircuit> on a saturday at peak hours
19:07 < phantomcircuit> gmaxwell, so in trying to improve the reliability of automated withdrawal request processing
19:08 < phantomcircuit> i've come to the point that i want to use raw transactions to effectively get a two phase commit
19:08 < gmaxwell> phantomcircuit: seperate the sign from the send and make it in the database before you send?
19:08 < phantomcircuit> currently what im doing is setting the transfer to processing, calling sendtoaddress and updating with the transaction result
19:09 < phantomcircuit> however if there is a failure after sendtoaddress but before the request is updated
19:09 < phantomcircuit> then i have to manually go in and fix it
19:09 < phantomcircuit> so my question is
19:09 < phantomcircuit> is there a way to get bitcoind to do the output selection
19:09 < gmaxwell> right, so you want a sendmany that returns a raw transaction?  then you can call sign on it, write it to your database.. then send it?
19:09 < phantomcircuit> gmaxwell, yeah
19:10 < phantomcircuit> well except im using sendtoaddress since there's very rarely transactions that can be grouped
19:10 < gmaxwell> yea but you can sendmany with just one output.
19:10 < phantomcircuit> yeah
19:10 < gmaxwell> I would have done that already except we @#$@# call signing inside the coin selection innerloop. Which is retarded. If you feel like fixing that, the rpc would be really easy to write.
19:11 < gmaxwell> Though I think it should just be a sendmanyraw or a flag to sendmany that lets it output the raw txn.
19:11 < nanotube> midnightmagic: scrollback in this channel. :)
19:12 < phantomcircuit> gmaxwell, well the returned tx would optimally be signed already
19:12 < jrmithdobbs> gmaxwell: that rommixmc thing is interesting and similar to my "fix" for scrypt that i haven't had time to work on since last i talked to you about it, haha
19:12 < midnightmagic> nanotube: thanks man
19:12 < phantomcircuit> signed but not committed to the wallet.dat database yet
19:12 < jrmithdobbs> gmaxwell: that is a very cool solution
19:13 < gmaxwell> phantomcircuit: for your usage, but not signing it is more general. What if your online wallet was locked.. and your unlocked wallet was not "online" ? e.g. just rs232 connected box or something.
19:13 < phantomcircuit> gmaxwell, ah
19:13 < phantomcircuit> yeah i guess that's true
19:13 < gmaxwell> phantomcircuit: the cost of not signing it is that you have to make another rpc roundtrip, pretty mild cost.
19:14 < phantomcircuit> well the primary cost is that i have to fix the coinselection stuff
19:14 < jrmithdobbs> gmaxwell: catenta still doesn't address the dependency on sha2 for the first obsfucation though unless i'm missing something =/
19:14 < phantomcircuit> where as right now i could probably add a flag to sendmany to not broadcast/save to the wallet
19:15 < nanotube> midnightmagic: starting about 2200 my time on sep 8. :)
19:15 < jrmithdobbs> gmaxwell: sorry, responding to something from like 2 days ago :)
19:15 < gmaxwell> yep. The problem isn't fundimentally hard. The signature will only have four possible sizes: compress, uncompressed, p2pubkey compressed, p2pubkey uncompressed,  (assuming that you just aproximate the size by rounding up.
19:15 < gmaxwell> jrmithdobbs: yea its fine I knew what you were responding to.
19:16 < gmaxwell> jrmithdobbs: I thought the recommendation on catenta was just use sha3 all ove.r
19:16 < jrmithdobbs> gmaxwell: their splitting of client/server work is very much in the same line of thinking i was going down, makes me wish I had time to go back to that before the PHC deadline cause that's nice confirmation i was on to something ;p
19:16 < Luke-Jr> jrmithdobbs: you have a "fix" for scrypt?
19:17 < jrmithdobbs> Luke-Jr: i have a set of improvements i've been toying about with for almost a year now, yes
19:17 < Luke-Jr> jrmithdobbs: does it make it a viable POW?
19:17 < jrmithdobbs> part of it was trying to address the cache timing attack the cantena guys address, in fact
19:17 < gmaxwell> and I was pointing catenta out to jrmithdobbs because I knew that he was concerned with some of the things it addresses.
19:18 < jrmithdobbs> gmaxwell: i don't see how sha3 is all that much better suited other than we don't know it's issues yet =/
19:18 < jrmithdobbs> i mean it's obviously better than using sha3 for the task
19:18 < jrmithdobbs> but ...
19:18 < jrmithdobbs> err better than using sha2*
19:18 < gmaxwell> jrmithdobbs: really any function is suited. Nothing busted in the last 20 years has been busted so much to harm its usage as a kdf.
19:18 < gmaxwell> (any cryptogrphic hash)
19:19 < jrmithdobbs> gmaxwell: that's true, md5 is still usable for that in most scenarios
19:19 < jrmithdobbs> not recomended, but realistically, it's usable
19:19 < gmaxwell> s/most/all/ really. md4 would be fine too.  Better to use something better... but.
19:20 < jrmithdobbs> gmaxwell: if you have access to enough key samples for some reason md4/5 could be problematic, no?
19:20 < gmaxwell> I don't think so, not after you've iterated them thosands of times.
19:20 < jrmithdobbs> but yes, anyways, i'll conceed your point ;p
19:20 < phantomcircuit> i love that most CA root certs are md2
19:20 < jrmithdobbs> that line of thinking just makes me feel dirty
19:21 < jrmithdobbs> because it reeks of situations like with the  people with "credibility" telling people to revert to 20-year-old-known-broken-to-statistical-analysis ciphers for a half a decade so the nsa can log all our traffic =/
19:21 < gmaxwell> I'm really excited about the asymetric memoryhard trapdoor proof of work I came up with this morning.. though I worry the validation will be too slow.
19:21 < jrmithdobbs> (fuckin rc4)
19:21 < gmaxwell> yea... wtf well.. ssl is a cluter@#$@ in general.
19:22 < jrmithdobbs> anyways, back to work
19:22 < jrmithdobbs> gmaxwell: i'm gonna read over that a few more times, there's good work there (catenta)
19:23 < phantomcircuit> damn
19:23 < jrmithdobbs> nice to see someone besides the scrypt guy looking at the problem. not enough people are
19:23 < phantomcircuit> my 2TB external hdd is full
19:23 < phantomcircuit> >.>
19:23 < gmaxwell> now the problem with catenta is that it's new. :(
19:23 < jrmithdobbs> gmaxwell: it's not though
19:23 < jrmithdobbs> gmaxwell: it's a modified rommix with a different hash
19:23 < gmaxwell> scrypt was finally getting old enough to get people to accept it, and now we have a new version.
19:23 < gmaxwell> I know.
19:24 < jrmithdobbs> true
19:24 < gmaxwell> I did read the paper. I love it. Its a big improvement IMO.
19:24 < jrmithdobbs> and scrypt wasn't "new" either
19:24 < jrmithdobbs> it was old stuff applied in a novel way
19:24 < phantomcircuit> that's new
19:24 < phantomcircuit> :)
19:24 < jrmithdobbs> scrypt was really just modernized bcrypt with more long term thinknig and a better cipher behind it, if you look at it
19:25 < jrmithdobbs> the basic construction isn't very novel (it's cool, don't get me wrong ;p)
19:25 < gmaxwell> well no, the romix idea was novel. Also catenta still doesn't go quite far enough.
19:25 < gmaxwell> e.g. it doesn't make optimal use of the memory hierarchy.
19:25 < jrmithdobbs> it also still reveals too much to the authenticating party imho
19:26 < jrmithdobbs> but that's a bigger problem
19:26 < jrmithdobbs> (I don't think the authenticating party should ever have the hash)
19:26 < gmaxwell> ideally such a function would achieve optimal speed only if you has a given ratio of adder speed to l2 cache speed to memory speed.
19:27 < jrmithdobbs> and once you start needing to think about that portability kind of goes out the window for the simple solutions
19:27 < jrmithdobbs> but isn't optimized portable code an oxymoron?
19:28 < jrmithdobbs> gmaxwell: also why do they recomend keccak and not blake2? if you're trying to avoid cache timing issues isn't reusing salsacore as much as you can one of the best things you can do?
19:29 < jrmithdobbs> if it's all re-referencing the same damned code the code doesn't get kicked out of cache, after all
19:29 < jrmithdobbs> or at least, it's harder to forcibly evict
19:30 < gmaxwell> it doesn't mater if its kicked out of cache, the access pattern is not data dependant.
19:34 < jrmithdobbs> oh bleh, that's right it's timing on the data access not code segments, i'm going to run instead of saying stupid shit on the internet in my prednisone fueled mania
19:34 < jrmithdobbs> ;p
19:34 < jrmithdobbs> and by run
19:34 < jrmithdobbs> i mean physically
19:47 < phantomcircuit> i just realized something
19:47 < phantomcircuit> i can just copy this desktops hdd into a vm
19:47 < phantomcircuit> derp
19:48 < phantomcircuit> obvious solution is obvious
19:48 < phantomcircuit> sorry totally off topic
20:23 < nanotube> anyone care to test if my bitcoind hidden service is visible? gb5ypqt63du3wfhn.onion
20:31 < gmaxwell> 2013-09-11 00:26:20 receive version message: version 70001, blocks=257216, us=5yljdotwhmx65nlk.onion:8333, them=gb5ypqt63du3wfhn.onion:8333, peer=127.0.0.1:58807
20:31 < gmaxwell> so you're working and sending out the right address in your version message.
20:32 < nanotube> cool. :)
20:32 < nanotube> i have your node in addnode
20:33 < nanotube> my guess is it isn't currently possible, but it probably should - to set connection limits separately for tor and non-tor.
20:33 < nanotube> to ensure an active tor-nontor bridge
20:33 < nanotube> otherwise given the relative paucity of tor nodes, it could be that all slots can be eaten up by nontor nodes and you lose the tor bridge
20:33 < nanotube> ?
20:33 < jrmithdobbs> nanotube: it's possible in that you can give a list of known nodes as -connect/-seed nodes
20:34 < jrmithdobbs> so long as all of them in your list don't drop at once ...
20:34 < nanotube> right, but let's say tor hiccups and you lose all tor connections, slots fill up...
20:34 < nanotube> then tor comes back up but you're cut fof.
20:34 < nanotube> off
20:34 < phantomcircuit> nanotube, connect reserves the slot for outbound connections
20:34 < phantomcircuit> (i think i should double check)
20:34 < nanotube> does addnode?
20:34 < jrmithdobbs> phantomcircuit: but on the other side the slots might have been taking is what he's saying
20:35 < nanotube> connect iirc says 'connect only to this node and nothing else'
20:35 < nanotube> addnode says 'add this to whatever else is going on'
20:35 < phantomcircuit> nanotube, connect does prevent connecting to anything else
20:35 < nanotube> so obviously connect would reserve.
20:35 < phantomcircuit> nanotube, i dont think what you're trying to do will work
20:35 < jrmithdobbs> sorry i meant addnode and it doesn reserve if i remember the code correctly
20:35 < jrmithdobbs> *does
20:35 < nanotube> ah so if addnode reserves, guess that's fine
20:35 < jrmithdobbs> but only on connecting-from (client) side
20:35 < gmaxwell> Addnode reserves.
20:36 < gmaxwell> yes only on the from side.
20:36 < jrmithdobbs> but if you have a pool of say 20 bridges that all -connect to each other's onions ... problem solved
20:36 < nanotube> ah, so the scenario of 'tor dies, slots fill up' is still a threat to tor bridging?
20:36 < jrmithdobbs> s/-connect/-addnode
20:36 < nanotube> at any rate, in the meantime, throw me your tor node addresses and i'll addnode them. :)
20:37 < jrmithdobbs> took mine down due to lack of interest/connections
20:37 < jrmithdobbs> months ago
20:37 < nanotube> hum.
20:37 < phantomcircuit> i wonder how much effort it would take to add a gui for adding reserved slot peers
20:37 < gmaxwell> jrmithdobbs: huh? mine usually has >30 HS inbounds...
20:37 < phantomcircuit> so normal people could connect to their friends (or at least try to)
20:38 < nanotube> forget gui, a config option would be nice. :)
20:38 < nanotube> iow, if addnode reserved a slot.
20:38 < jrmithdobbs> gmaxwell: mine didn't and "months" is actually almost a year now
20:38 < phantomcircuit> nanotube, if gmaxwell says it does it probably does
20:38 < gmaxwell> well you can't reserve HS inbound for specific HS peers sadly.
20:38 < phantomcircuit> :)
20:38 < phantomcircuit> oh i didn't mean inbound
20:38 < nanotube> he says it doesn't
20:38 < phantomcircuit> reserved inbound slots isn't important
20:39 < gmaxwell> nanotube: addnode outbound always works.
20:39 < nanotube> anyway just a suggestion. it's a bit of an edge case.
20:39 < phantomcircuit> unless you're super popular you're not going to hit the 128 limit
20:39 < gmaxwell> or dos attacked.
20:39 < nanotube> i don't have the ram for 128, i'm running with 16 :)
20:39 < jrmithdobbs> phantomcircuit: the node i turned tor off on had 512 max cons with ~300-380 constant non-tor and 5-10 unconnectable tor nodes
20:39 < nanotube> will see how it behaves and maybe up it a bit
20:39 < phantomcircuit> gmaxwell, shrug
20:39 < jrmithdobbs> (plus sipa and gmaxwell's tor nodes)
20:39 < gmaxwell> nanotube: since 0.8.1+ you shouldn't need a lot of ram for a lot of inbounds.
20:40 < nanotube> so going from 16 to 128, what's the impact?
20:40 < phantomcircuit> gmaxwell, possibly select should be replaced with epoll()
20:40 < gmaxwell> nanotube: dunno, haven't measured lately. if this is a non-wallet node running the disable wallet patches will also save you 50 mb.
20:40 < phantomcircuit> nanotube, select() is slower but only marginally so and you're using up file descriptors
20:40 < nanotube> currently i have 268/585 res/virt ram use with 16
20:41 < phantomcircuit> nanotube, running what version
20:41 < nanotube> latest release .8.4
20:41 < phantomcircuit> also have you synced
20:41 < nanotube> yes
20:41 < phantomcircuit> this have an active wallet attached to it?
20:41 < nanotube> well, not active, it's the empty default wallet
20:41 < gmaxwell> I'm 262mb res, but at the moment I only have 11 peers.
20:41 < phantomcircuit> weird
20:42 < nanotube> well, i'll run at 16 for a day or two and see how it is, then up it to 128 and see what it does.
20:44 < phantomcircuit> blargh debian ftp mirror rate limiting me
20:45 < nanotube> hm, guess the network has plenty of open slots - i'm running at 15 connections heh
20:46 < phantomcircuit> nanotube, it's exceptionally random how many inbound connections you'll get
20:46 < nanotube> or maybe my fresh node hasn't yet been discovered by much of the network.
20:46 < nanotube> mm
20:47 < phantomcircuit> nanotube,     "connections" : 81,
20:47 < phantomcircuit> very long lived node
20:47 < nanotube> nice
20:48 < nanotube> 2013-09-11 00:47:14 Warning: Local node 127.0.0.1:36029 misbehaving (delta: 0)! heh well and there's some indication that i have tor peers. :)
20:50 < gmaxwell> nanotube: what I want to do for inbound is this something like this: Once every few minutes: If your inbounds aren't full, do nothing. If your inbound is full select a peer to evict with an algorithim like this:
20:50 < gmaxwell> Remove addnoded peers that we're not also outbound to from consideration.
20:50 < gmaxwell> Protect up to 8 longest connected localhost / local subnet connections.
20:50 < gmaxwell> Protect 10% of the remaining peers, ortered by most useful to us (e.g. most times the first inv for a new good block)
20:50 < gmaxwell> Protect 10% of the remaining peers, ordered by the lowest ever minimum ping latency, iff they have the useful flag, limited to one peer per netgroup.
20:50 < gmaxwell> Protect 10% of the remaining peers, ordered by H(secret IP), iff they have the useful flag, limited to one peer per IP.
20:50 < gmaxwell> Protect 10% of the remaining peers, ordered by longest connected
20:50 < gmaxwell> sort the rest by connection time divided by the number of peers on the same ip, select one to kick randomly weighed to pick short connections.
20:51 < nanotube> well according to netstat, i have 4 tor peers. \o/
20:52 < nanotube> ah so the idea is to prevent a node from being too static in the network, ic
20:53 < nanotube> i guess tor nodes would fall under localhost connections
20:53 < phantomcircuit> nanotube, getpeerinfo rpc call
20:54 < phantomcircuit> gmaxwell, instead of doing it when the slots are full, accept 129 connections and then select a peer to evict
20:54 < gmaxwell> nanotube: actually what I'd like to do is split this all by netgroup first, and handle tor peers totally seperately. e.g. move tor inbound to another port to distinguish it.
20:54 < phantomcircuit> i actually have a patch that does this which makes the simplest slot filling problems disappear
20:54 < gmaxwell> phantomcircuit: I think thats not right, in fact! because then someone connecting really fast can quickly use up all your probablistic slots.
20:54 < phantomcircuit> (magic)
20:55 < phantomcircuit> gmaxwell, well of course the disconnected slot could be the newly connected peer
20:55 < gmaxwell> and N in my example should actually be an exponential random variable.
20:55 < phantomcircuit> ie you could give a lot of weight to connections to your probabilistic slots which are new
20:55 < gmaxwell> phantomcircuit: I suppose that would be fine, or adjust the weigh-for-lowest duration to strongly prefer them. Fair enough.
20:56 < gmaxwell> In any case each of my "protect" groups is based on something which is hard for an attacker to fake.
20:56 < phantomcircuit> otoh i was actually considering randomly evicting peers even when you're not full
20:56 < phantomcircuit> just to churn the network and make it harder to do latency based analysis
20:57 < gmaxwell> Being on your local subnet, being net-geographically close is unfakable,  giving useful data (blocks) is not really fakable, having an IP that meets our secret criteria is only fakable with great expense, being connected for a long time is harder to 'fake'.
20:57 < gmaxwell> phantomcircuit: for outbound I think we should churn, for inbound I think not.
20:58 < gmaxwell> for outbound sipa has a proposal that randomly changes peers with a weight that prefers to evict the shortest connection. I suggested augmenting it by always keeping— say— the two most useful peers, so you don't rotate yourself into a useless partition of the network.
21:15 < nanotube> phantomcircuit: getpeerinfo - thanks. :)
21:16 < nanotube> if the shortest connections always get churned, they never get a chance to become long connections.
21:16 < nanotube> so some clients may forever be relegated to the churn pile?
21:19 < nanotube> i don't suppose it is possible to change maxconnections without restarting the client?
21:20 < gmaxwell> nanotube: unless they end up in one of those several protected classes I gave.
21:21 < nanotube> which as you yourself said, is not easy to do intentionally.
21:22 < nanotube> i guess eventually it'll end up somewhere which is relatively close with a low ping...
21:23 < nanotube> but wouldn't that make the bitcoin network more closely mirror physical geography
21:23 < gmaxwell> nanotube: thats also why it would only be a limited number of connections in that class. All those other connections are getting used by someone.
21:24 < nanotube> yea just wanted to make sure that a new node doesn't have trouble getting stable peers.
21:24 < gmaxwell> wrt becoming the longest, thats the idea of it being random though— it wouldn't punt the shortest life, it would say have a x% of the shorest a x/1.5% of the next... etc.
21:25 < nanotube> ah so weighted, not absolute, ok
21:26 < gmaxwell> yea, absolute would be broken. Sipa did some simulations for the weights for outbound and got some nice properties.
21:27 < nanotube> cool
21:27 < gmaxwell> but absolutely, the latency bias is dangerous if carried too far: you can get networks that self-partition.
21:27 < nanotube> right
21:27 < gmaxwell> Thats one reason I enumerated a class the H(secret+ip) one which is purely random, and not at all uptime or latency sensitive.
21:28 < nanotube> \o/ for randomness. :)
21:32 < nanotube> if nodes are going to be penalized for low uptime, could be a good idea to allow changing conf parameters without restarting the node via rpc. like bitcoind maxconnections X
21:35 < gmaxwell> at some point I believe we'll add some (or multiple) kinds of finite resource priority peers can use to get slots if they're having problems. I've got a couple ideas for that.
21:49 < nanotube> hm, that's interesting
21:52 < nanotube> mem was stable at 16conn, restarted with 128.
--- Log closed Wed Sep 11 00:00:26 2013