@quest
edited
more from that month
Quick Wallet Zero Balance Fix
Currently my wallet is showing $0.00 even though there are definitely funds in the wallet. When I open the console I see: `app.js?id=f819f3d5dce4d53eec1c:119 Uncaught (in promise) We're getting UTXOs with incorrect block height (2147483647 > 629315 - current block height), see https://read.cash/forum/readcash/wallet-balance-709ec680` Seems that `bchd.fountainhead.cash` is returning invalid results. I encourage the read.cash developers to point the wallet at `bchd.greyh.at` which is run by the BCHD team. This would immediately resolve the issue and does not have rate limits. Let's get this fixed today! User experience matters and this has plagued users for too long! This site is amazing and I don't want it to lose traction over such an easy issue to fix! UPDATE: Apparently there were other fixes needed and this wouldn't have actually been enough for the read.cash team. Fortunately the issue is now resolved, even though it took some time to get there. =)
11 comments
It sporadically works for me...
Yeah, with this fix it would work 100% of the time. =)
Going to link to this in forums
@quest We and the fountainhead team notified the BCHD team about a bug that causes this and the issue was immediately dismissed and closed (basically: WONTFIX). https://github.com/gcash/bchd/issues/351 I don't think after this I can trust the claim that "This would immediately resolve the issue". There is a bug in BCHD that needs to be found and fixed. It is a data loss bug that was dismissed... that doesn't build the trust. This is one reason why we will stay with fountainhead, so that this bug is visible and not dismissed. And there is a second problem. We need to double check the upvotes on the server side so that we don't just trust users that "we've paid this, ok?". Our server-side language is PHP. PHP doesn't have any good support for GRPC. The only library that provides via PECL it adds 30+ minutes to our deploy time, which is totally unacceptable. Fountainhead team provides a HTTPS proxy for us. Even if we switch to greyh.at we would still have to use fountainhead's proxy, which proxies their node, so we'll get two different data sets. So, it's not a "quick fix".
This was closed because the node operator posted in the thread that their node had unsafely shut down and they claimed that was what caused the error. Perhaps after they fixed their node you could have just replied that the issue was not fixed and users were still experiencing issues. It looks like the issue is you checking mempool transactions that haven't confirmed yet. The number of confirmations returned should definitely be fixed and return 0. I can get this fixed for you. As for the height ... There needs to be some integer value there. That value probably won't change. Sorry Chris closed the issue so quickly, but when the issue came back you should have spoken up. Next time I won't bother to suggest any quick fix. It appears you don't appreciate working with us.
Ok, there are a few issues mixed here. 1) Unsafe shutdown should not lead to severe data corruption in node software. I mean, node software is basically a database and pretty primitive at that. There should be something like append only log or something like that, that even in case of abrupt shutdown it should maybe be behind, but not lead to full node reinstallation. I was really-really surprised when this was dismissed. Fountainhead team (as far as I know) to resync BCHD. This is unacceptable for production software. I mean... power failures happen. It should not lead to corruption. That's why I strongly believe this must be fixed, not dismissed. 2) "Sorry Chris closed the issue so quickly, but when the issue came back you should have spoken up." Well, this (your post and my comment) our interaction was the first time this issue came up to our attention. I've linked it in GitHub. I'm not sure what else could I have done to "speak up". 3) "Next time I won't bother to suggest any quick fix. It appears you don't appreciate working with us." I haven't worked with you, my GitHub interaction with the first time. I must admit it was pretty disappointing, because I always seen BCHD team as very professional. Disregarding data loss issue was... unsettling. I ignored it. Then I responded why your quick fix isn't a fix for us at all, since we need an HTTP(s) proxy (I explained why). Imagine this - this issue is not fixed, we switch to bch.greyh.at, but still use HTTP proxy from fountainhead, which still experiences data loss due to the bug. So, none of the upvotes really work, since we can't confirm them. Do you see why this is not a quick fix for us? It's not like we're annoyed, it just won't work until either the bug is fixed or BCHD has a REST interface (and I believe I read somewhere that BCHD team won't implement REST and PHP doesn't have good GRPC), so we're kind of at an impasse here. Right now we're swamped with support requests about missing upvotes/payments/etc... And frankly we have no idea what to do. We have to manually check everything that happened, verify transactions and sometimes even reimburse people for money they think that they got, but they didn't. We do it with our own money, because we understand that people feel read.cash cheated/failed them. Again, even if we switched to bch.greyh.at - it won't help, as described above. One more thing to add, why I believe that BCHD should fix bug [351](https://github.com/gcash/bchd/issues/351) instead of ignoring it: fountainhead is a very professional team running BCHD. If they can't run it without constantly re-syncing it, how would expected millions of users run it? We're more than willing help you research it, but we'll never understand anyone ignoring a data corruption issue in a database-like software.
We are very aware of the issue that hit the fountainhead.cash team. We have recovery code for improper shutdowns, and most of the time it works just fine, we have had a very hard time reproducing the error case. It has hit me as well, and trust me, we aren't happy that this happens. We have combed over the code many times, and the recovery does work a significant amount of the time. Unfortunately that is one of the most complex parts of all the code in bchd. It would be great to get more eyes on the issue, but we absolutely have not "ignored a data loss issue". Sorry you had a hard time with the return values on mempool transactions. I will fix the confirmation value so its 0, however for the block height there is no null for integers with grpc. That means we have to return some numeric value. Personally I think -1 will probably work. Anyhow, we never ignored a data loss issue. Just because we didn't track it on your issue, doesn't mean it is ignored. Your assumption that we would just ignore it is quite surprising.
I'm glad to hear that this is being worked on. Here's why I assumed it was not: "There is code to do it but there is a bug in it that causes it to sometimes fail that is very difficult to figure out. cpacia closed this" I mean, there's no mention that this is an ongoing research, no link to any other open issue, just "it's difficult, closed". Having worked far too long in the corporate world for me it sure sounds as "don't care, won't fix". If Chris linked to an open issue tracking it, it would've made a lot more sense. Anyway, I'm glad to hear that this is being worked on, but I think we'll still prefer to stay with fountainhead, because the guys are pretty professional and our usage help surface any hidden problems with BCHD, which is good, because it makes BCHD more resilient. And as I've previously mentioned - we're very dependent on the REST interface, which I hope to see in BCHD someday supported natively. "-1" for the block height is a great idea.
Trust me, we have been looking at that issue. We really do care and have discussed this with JT and other folks at fountainhead.cash. They do great work, and working with them is excellent if you need additional items like the proxy. That being said, it would be great to also offer this on the BCHD teams official services. We would happily also provide a proxy, just need some info on what you need so you can have a backup just in case something goes wrong. I will try to update those mempool values soon. Want to make sure things are as easy to use as possible for developers. Love the site, glad we were able to talk this out.
Frankly, I think having an official REST interface inside of BCHD that just proxies GRPC would be more than enough. (grpc-web, maybe?) Maybe as an optional feature. We only use one function (GetTransaction) which is proxied by Fountainhead using something like Swagger, so basically, one-to-one translation. But frankly a lot of other functions could be using. I believe it's a non-official interface that they don't officially advertise, so I can't point out the details of where it is. Basically we send something like `{"json": {"hash": "...."}}` and get the details back as JSON. You can probably ask them for the details. Thanks!
> As for the height ... There needs to be some integer value there. That value probably won't change. The problem is that when we see height of "2146855420" we assumed that was the manifestation of the bug, but in reality it means "unconfirmed". A "null" value would have been probably much intuitive. We have removed this check. It seems to be working now. So, we believe that second corruption probably hasn't happened, but it was our misunderstanding of the number 2146855420 as the height. (Personally, I think it should be concerning, since new BCHD GPRC consumers might be just as confused as we are about this number)