Dark secrets of the Grasberg DAA
On July 23, 2020, Bitcoin ABC announced Grasberg, a new difficulty adjustment algorithm (DAA) for Bitcoin Cash, to much fanfare. The announcement certainly sounds exciting, and mentions several advantages to the new DAA. But the announcement is more noteworthy for the facts that it omits. There are some very important facts about the Grasberg DAA that Bitcoin ABC did not mention. The Bitcoin Cash community should be aware of these facts before making a decision about whether to accept Grasberg, or whether to choose the alternative instead. https://blog.bitcoinabc.org/2020/07/23/announcing-the-grasberg-daa/
In this article, I will present evidence for four types of arguments against Grasberg:
Grasberg will have *negative practical effects* on Bitcoin Cash (sections 1-2)
Grasberg will *compromise the economic principles* of Bitcoin Cash (sections 2-3)
Grasberg is a *dangerous political tool* of Bitcoin ABC (sections 3-6)
Grasberg is a *technically inferior* *option* to fix the DAA issue (sections 7-10)
In this article, I present a large number of serious claims against Bitcoin ABC. These claims require verification. But **serious claims demand severe action.** If even a few of my claims are accurate, then Bitcoin Cash should respond quickly and strongly to respond to this threat.
The first few of Grasberg secrets concern the mechanism for correcting historical drift. Bitcoin ABC did not clearly state what the costs of this system would be, and they are significant.
1. Grasberg will cause 11.25 minute block times for the next 6.5 years
Grasberg will cause blocks to be mined 12.5% slower until April 2027. This means:
Confirmations will be 12.5% slower
Deposits to exchanges will be 12.5% slower
The 50-tx chain limit will be 12.5% more restrictive
BCH miners will earn 11.1% less revenue
BCH will have 11.1% less hashrate and security
Height-based nLockTime transactions will be unlocked 12.5% later
The 11.25 minute block times is not a mistake. It is the intended effect of Grasberg's `computeTargetBlockTime(...)` function. Bitcoin ABC knew that Grasberg would produce 11.25 minute block times, and they chose not to mention it in their announcement.
2. Grasberg will change the BCH coin emission schedule
Bitcoin ABC claims that the main motivation for Grasberg is to avoid redefining the emission schedule. They claim that this is the main reason why ASERT can not be an option. To quote their article:
*Choosing a reference point other than the genesis block is effectively equivalent to redefining the coin emission policy, which is a big NO."*
Unfortunately, they have it backwards. The only way to avoid redefining the coin emission schedule is to use the *most recent* (pre-fork) block as a reference point. This is what my ASERT proposal does, but it is not what Grasberg does.
Most people who are not experts in DAAs (and ASERT in particular) may find this terminology confusing and unfamiliar, and may find a theoretical argument difficult to follow. Fortunately, there is much simpler way to explain this issue: with data. Data don't lie.
We can see which algorithm actually redefines the coin emission schedule by looking at the coin issuance schedules for Grasberg, ASERT, and the current status quo (cw-144).
The current coin emission schedule is **900 BCH per day** (6.25 BCH * 144) until the next halving.
The coin emission schedule for ASERT is also **900 BCH per day** until the next halving. ASERT preserves the existing coin emission policy, and is nearly indistinguishable in coin emission from the current DAA (cw-144) without zooming in. That's because ASERT is designed to use the last cw-144 block as its reference; ASERT simply extends the current coin emission policy, but with greater precision and without the 24.5-hr oscillations. http://toom.im/bch/grasberg_coin_issuance.html
The coin emission schedule for Grasberg is **800 BCH per day (**6.25 BCH * 144 /112.5%) until the next halving. Grasberg defines an entirely new coin emission policy. Grasberg generates blocks 12.5% slower, which gives us 11.11% fewer coins for the next 6.5 years, after which it delays all halvings by about 8 months. This new, unexpected coin emission schedule is defined by several new, arbitrary numbers. Nowhere in the Bitcoin white paper or original source code was the number 12.5% or 675 seconds listed. The consequent 6.5 year length is likewise arbitrary. These numbers were chosen by Amaury Séchet and Bitcoin ABC without explanation. The 12.5% number in particular is eerily familiar. https://medium.com/@jiangzhuoer/infrastructure-funding-plan-for-bitcoin-cash-131fdcd2412e
If Bitcoin Cash is to be hard money, then it must resist attempts by developers to arbitrarily change the coin emission schedule. If we allow Amaury Séchet to change the schedule now, that sets the precedent that changing the timing or quantity of coin issuance should be within his power. It makes him akin to a central banker or a chancellor, handing out free money to hodlers. With that precedent, and when this change expires in 6.5 years, we might find that Bitcoin ABC's chancellor is on the brink of a second bailout. https://explorer.bitcoin.com/bch/block/000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
Choosing the genesis block as a reference point is effectively equivalent to redefining the coin emission policy from scratch, which is a big NO.
3. Grasberg crosses the line from technology to politics
The coin emission schedule determines how new wealth is distributed. Changes to the coin emission schedule is changing the distribution of wealth. This is a major red flag.
Changes like Grasberg's 11.1% reduction in coin issuance rate are extremely dangerous precisely because they are attractive to many people. It's easy to rationalize Grasberg as benefiting hodlers via a 11.1% reduction in issuance and inflation. A person who already has a large stash of BCH -- for example, if they had mined a lot of BCH during Aug 2017-Nov 2017, when miners were earning up to 3.5x as much BCH per day as Satoshi intended -- then they might be interested in seeing a reduction in issuance to give them a short-term boost in their investment's value and make it easier for them to cash out.
We should not let ourselves be swayed by arguments like these, because coin issuance is inherently a zero-sum game. For every dollar of direct benefit that a coin issuance change gives to old users and hodlers by allowing them to sell at a temporarily higher price, there is a dollar of direct detriment to new users by forcing them to buy at a temporarily higher price. https://en.wikipedia.org/wiki/Zero-sum_game
Furthermore, any attempt to manipulate the value of the currency by changing inflation is bound to backfire. By changing coin issuance this one time, Bitcoin ABC would be sending a strong signal that the current economic and monetary rules of Bitcoin Cash are not permanent, and that any coins they buy are for a supply that is subject to change. Current holders will lose faith in the currency's future, and will be inclined to sell while the price is still high. Potential buyers will lose faith in the currency's future, and will be less inclined to buy. A currency is only as strong as its promise of future value.
Bitcoin is supposed to be censorship-resistant worldwide electronic cash, to bring economic freedom to the world, and to free people from the economic control of corrupt and self-serving governments, not to be an investment vehicle for early adopters to get rich. In order to accomplish its goals, Bitcoin's rules need to be kept fair and economically consistent. We should not allow changes to Bitcoin Cash's economic policy that take from future users to enrich the present ones. If we follow this path, there likely won't be any future users to take from.
4. Grasberg's historical drift correction is unpopular among BCH developers
On Monday, July 26th, a development meeting was hosted by Future of Bitcoin Cash on the BCH DAA issue. The entire 1h50m meeting was dedicated to the historical drift correction issue. This meeting was attended by the following ten developers: https://www.youtube.com/watch?v=F5b0Unf1aA0
Amaury Séchet, the lead developer of Bitcoin ABC
Antony Zegers, a developer of Bitcoin ABC
Zawy12 (Scott Roberts), a non-BCH developer who specializes in DAAs
Jacob Eliosoff, a non-BCH developer who specializes in EMAs, and who was the first to discover ASERT
Jonathan Toomim (myself), an independent BCH developer
Jason Dreyzehner, a developer of an SPV wallet at bitauth.com
Chris Pacia, one half of the BCHD team
Josh Ellithorpe, the other half of the BCHD team
Andrew Stone, the lead developer of Bitcoin Unlimited
Freetrader, the lead developer of BCHN
After having discussed the drift issue for nearly two hours, an informal poll was done of the developers about whether or not they supported the proposal to do historical drift correction. The results were:
2 in favor (Both Bitcoin ABC devs)
2 abstentions (both non-BCH devs expressed criticism, but did not vote)
6 opposed (everyone else)
Only 8 BCH developers were present in this meeting, but sentiment among the broader development community seems similar. Some of the developers who have made statements opposing historical drift correction include:
Tom Zander of Flowee https://www.reddit.com/r/btc/comments/i2a6ir/bitcoin_cash_historic_and_forecasted_drift/g0386zl/
Jonathan Silverblood of BU and Cashaccounts https://bitco.in/forum/threads/buip150-reject-past-drift-correcting-daas.26486/
Mark Lundeberg https://twitter.com/BitcoinUnlimit/status/1288177166102999053
Jacob Eliosoff (though he abstained during the meeting) https://medium.com/@jacob.eliosoff/drift-correction-and-the-asert-daa-2c2d148c3250
Josh Green and of Bitcoin Verde (4 other companies also signed) https://read.cash/@GeneralProtocols/joint-statement-on-aserti3-2d-algorithm-f98f0a2c
BigBlockIfTrue of BCHN https://read.cash/@BigBlockIfTrue/bitcoin-cash-historic-and-forecasted-drift-36f8a6d4
Calin Culianu of BCHN https://read.cash/@NilacTheGrim/hard-money-sound-money-war-is-peace-day-is-night-black-is-white-1984-levels-of-double-speak-b1807d9a
Besides Amaury and Antony, those in favor of drift correction, that I'm aware of, include:
Joannes Vermorel (via the [WG] Difficulty Adjustment group on Telegram)
The fact that this proposal appears to be supported only by Bitcoin ABC, and garners strong and near-universal opposition from other developers, suggests that ABC's attempt to push Grasberg's historical drift correction through in the November upgrade could result in a chainsplit.
5. A concrete proposal had reached ABC at the time of their writing
ABC's Grasberg announcement began with the following paragraph:
*While there is a lot of discussion around the DAA, no concrete proposal has reached ABC at the time of this writing, which does not leave us with sufficient time to review adequately, simulate and test, and get feedback addressed before the feature freeze on August 15th.*
**That claim is simply false.**
I submitted my team's proposal for aserti3-2d on July 8th at 04:55 UTC -- a full 15 days before Bitcoin ABC announced Grasberg. Our proposal, aserti3-2d (a cubic integer approximation of ASERT with a 2-day half-life), was very concrete, and included both a Python3 implementation and a C++ implementation built on top of the BCHN full node implementation. https://read.cash/@jtoomim/bch-protocol-upgrade-proposal-use-asert-as-the-new-daa-1d875696 https://gitlab.com/jtoomim/bitcoin-cash-node/-/commit/fd92035c2e8d16360fb3e314b626bf52f2a2be67#ee35cfc09ad35fa12b8d0a6280a950e4edde3d20_285_299
When Bitcoin ABC made the above statement, they were flooded with comments that called out the falsehood on Amaury Séchet's read.cash article and again on Reddit. Bitcoin ABC has not apologized for their error. https://read.cash/@deadalnix/announcing-the-grasberg-daa-ff52e96d#comment-4bf763a6 https://read.cash/@deadalnix/announcing-the-grasberg-daa-ff52e96d#comment-d78eca6c https://read.cash/@deadalnix/announcing-the-grasberg-daa-ff52e96d#comment-937e90ce https://www.reddit.com/r/btc/comments/hwkeqq/announcing_the_grasberg_daa/fz0970g/
Our July 8th proposal and implementation had attracted immediate attention and code review from many BCH node developers, including Flowee (Tom Zander), Knuth (Fernando Pelliccioni), BCHN (freetrader and mtrcyz), and Bitcoin Unlimited (Andrea Suisani). The proposal garnered a lot of excitement, general approval, and collaboration from most of the BCH development community, and several nodes have already begun integrating the code. However, we did not get any code review from any ABC developers.
I sent links to my code to two of ABC's developers (Antony Zegers and Amaury Séchet) several times, but they never tested or reviewed the code. From July 13th-15th, Amaury Séchet asked me some questions about the contents and design of the `aserti3-2d` algorithm and code, which I answered promptly and fully. https://twitter.com/KnuthNode/status/1290237886911188992 https://gitlab.com/jtoomim/bitcoin-cash-node/-/commits/daa/aserti3-wip https://github.com/jtoomim/bitcoin-abc/commit/83de74761631e509cba3fd58455b9f094e219a8c
The purpose of these questions does not appear to have been to help them perform code review, or to adequately simulate and test my proposal. They didn't have time to do that, *because they were too busy secretly developing Grasberg*. A lot of Grasberg's design and features is based on my team's research, or is directly derived from my implementation. This includes the fundamental premise that the DAA is broken, that a new DAA can solve the issue, that Bonded Mining is not an appropriate choice, that ASERT or RSERT is likely to be the best choice, and the use of the `2^(x+n) = 2^(x) * 2^(n)` identity to help with the integer approximation step. https://www.reddit.com/r/btc/comments/fanc6o/the_bch_difficulty_adjustment_algorithm_is_broken/ https://old.reddit.com/r/btc/comments/hph3gk/review_of_the_asert_daa_proposal/fxrti0e/
I am not bothered by the fact that I did not receive as much credit for my work as they should have given me. What bothers me is that Bitcoin ABC *stonewalled* my proposal and denied its existence, while simultaneously copying its critical features, and finally attempted to take primary credit for fixing the DAA and the oscillation issue.
Furthermore, their announcement provided Grasberg as a done packaged deal, a *fait accompli*, rather than as a proposal:
Bitcoin ABC is therefore moving forward with the Grasberg DAA.
I try to always assume good faith in my interactions with other developers in open source projects, but I struggle to explain this behavior as being in good faith. These are the facts:
Bitcoin ABC was aware of my proposal
Bitcoin ABC asked me questions on my proposal
Bitcoin ABC publicly claimed the nonexistence of my proposal
Bitcoin ABC developed a competing project, using features from my proposal
Bitcoin ABC's project shared fundamental properties with and was influenced by my proposal
Bitcoin ABC claimed they did not have time to review, simulate, test, and get feedback addressed for my proposal or any other proposals
Bitcoin ABC had time to *develop*, review, simulate, test, solicit feedback, and address feedback for their own proposal, even though it was started much later and was less mature than mine
Bitcoin ABC did not announce their project as a proposal, but as a conclusion
Given these facts, I think it is clear that at a minimum, Bitcoin ABC's behavior has been unprofessional and immature. However, it's hard to explain these facts based solely on immaturity. Consequently, I think the BCH community needs to consider the hypothesis that Bitcoin ABC has been acting dishonestly and selfishly.
The most charitable explanation I can give for this behavior is that Bitcoin ABC has a severe case of Not-Invented-Here syndrome (NIH), which causes Bitcoin ABC to prefer to spend extra time developing their own ideas instead of reviewing other's code. However, NIH fails to adequately explain why Bitcoin ABC did not adequately acknowledge the *existence* of another proposal, nor why ABC presented Grasberg as a *fait accompli*. This suggests that a less charitable explanation may be more accurate. https://read.cash/@noise/the-not-invented-here-syndrome-in-bitcoin-cash-development-0f983972 https://www.reddit.com/r/btc/comments/i0bvo7/this_is_what_not_invented_here_syndrome_sounds/
To quote developer Andrew Stone of Bitcoin Unlimited: https://youtu.be/F5b0Unf1aA0?t=4250
A greater acknowledgement of the previous work done is probably very important. And in particular in this case to be branded as a whole new algorithm on the outside, but if you look under the covers, it's what Jonathan has been working on for a long time plus some other stuff, you know, **makes it look like a steal...**
We should consider the possibility that Bitcoin ABC was attempting to copy the core elements of the design of my team's DAA fix in order to take credit for the work and to reaffirm the political power that they had lost after their failed IFP proposal. The historical drift correction proposal and new coin issuance schedule may have been a concession or political compromise to one of their allies or funding sources intended to secure their continued support, or it may have simply been intended as a wedge to divide the community and allow them to seize control with decisive leadership.
I do not know what Bitcoin ABC's true motivations were. I hope they were for the best. Ultimately, it does not matter: Although we can hope for the best, we must always be prepared for the worst.
6. Grasberg is a big step on the path to corruption
Bitcoin was created largely in response to the corruption in the world's financial system that led to the 2007-2009 financial crisis. Bitcoin was designed to be resistant to government manipulation, censorship, and corruption. Bitcoin's publication was so long ago that many people have forgotten *how* Bitcoin achieved these properties, and have assumed that Bitcoin Cash will always have them. This is not guaranteed, and we are dangerously close to losing them.
Bitcoin's resistance to manipulation and corruption came as a result of a few properties of the system:
Bitcoin's main economic parameters were defined by math and code, and were thought to be immutable.
The system was not owned or controlled by anybody; there was no person who could be coerced.
Bitcoin Cash does not have those properties any longer. We have a hard fork every 6 months as directed by a "benevolent" dictator.
Grasberg takes us two steps further. If Grasberg is activated, it will demonstrate two new facts:
The main economic parameters of Bitcoin Cash, like the coin emission schedule, can be intentionally changed.
Amaury Séchet and Bitcoin ABC are in control of Bitcoin Cash, and have the sole and ultimate authority over protocol changes.
This makes Bitcoin Cash vulnerable to bribery and coersion. It tells any interested parties that if they want to change the core economic parameters of Bitcoin Cash, all they need to do is to get sufficient leverage against Amaury Séchet -- either via bribery or via a $5 wrench attack. https://xkcd.com/538/
It does not matter if Bitcoin ABC is currently corrupt. It does not matter if they were bribed or coerced into making this change. If we want our economic system to be resistant to corruption, we need proactive security, not reactive security. We need to prevent any actions that *could* be the result of bribery, coersion, or corruption. And we need to prevent any actions that would *encourage* them, or *enable* them.
Consider the following hypothetical scenarios.
**Scenario A:** Let's say that there is one or more large miners who mined a lot of blocks during the EDA era, and now has a large stash of coins that they want to appreciate in value. This miner may also wish to have greater control over the Bitcoin Cash blockchain, so that they are able to police it more effectively. This miner might advocate for a reduction in the BCH block issuance by 12.5%. This would not adversely affect the miner's *revenue*, since they can just switch some of their hashrate over to mine BTC instead. Since BCH gets only 2.5% of the total SHA256 hashrate, they would lose only 11.11% * 2.5% = 0.27% of their revenue. Meanwhile, their BCH holdings might appreciate 11.11%. If that individual or company has $200 million at stake, they might expect to gain returns of should their preferred policy be enacted. That gives them $21.67 million of benefit if they should succeed. Let's say that by donating $1.8 million to the reference implementation, this person has a 25% chance of gaining enough lobbyist influence in order to get his preferred change enacted. This results in an expected return on investment of $3.6 million for the miner and $1.8 million for the developer. https://blog.coinbase.com/a-deep-dive-into-the-recent-bch-hard-fork-incident-2ee14132f435
It's natural for individuals, businesses, or governments to want to make these changes to Bitcoin Cash. Blaming miners for wanting to get rich will not help us. If Bitcoin Cash is to be kept uncorrupt and free, we need to ensure that they *can't make them*.
**Scenario B:** Let's say there is a company, individual, or government with a significant financial interest in seeing a specific cryptocurrency platform collapse. Perhaps they are a major BTC or BSV holder, and expect their other cryptocurrency to gain market share in that event. Or perhaps they are a major bank and see cryptocurrencies as a threat. If they have $1 billion at stake, and they stand to gain 10% if they destroy Bitcoin Cash's market share, that's $100 million of potential gain. Let's say that by donating $2 million in funding a development team with a history of divisive behavior, and demanding a controversial policy be implemented, this entity has a 5% chance of triggering a chainsplit that would fragment the community, stagnate growth, and eliminate Bitcoin Cash as a threat. This results in an expected return on investment of $3 million for the attacker and $2 million for the developer. Both parties can improve their income by shorting BCH prior to the policy's deployment.
In both of these scenarios, there is no way for the general public to know that such an arrangement has been made as long as all parties involved cover their tracks appropriately. We should not wait for proof of malfeasance. The fact that these attacks are possible is reason enough why we need to act now.
7. Grasberg was not properly simulated https://read.cash/@jtoomim/dark-secrets-of-the-grasberg-daa-a9239fb6#7-grasberg-was-not-properly-simulated
In February, I did two weeks of development work on improving and extending a good multi-coin mining simulator to test out different DAAs and show conclusively that a DAA change could fix BCH's hashrate oscillation issues. I spent most of June again working with that simulator along with Zawy12 and Jacob Eliosoff to compare all known candidate DAAs to identify which ones had the best balance of performance, simplicity, and attack resistance. I consider detailed and well-documented simulations to be critical for any serious protocol change proposal. https://www.youtube.com/watch?v=Fd6GFpZjLxU https://github.com/zawy12/difficulty-algorithms/issues/62 https://read.cash/@jtoomim/bch-protocol-upgrade-proposal-use-asert-as-the-new-daa-1d875696
The Grasberg announcement said this about Grasberg's testing:
Bitcoin ABC has already run simulations and real-world testing of this algorithm
Several developers in the DAA working group on Telegram asked for a copy of the simulations that Bitcoin ABC claimed to have done on Grasberg.
All we got from Amaury was this spreadsheet. No explanation, no description, no justification, no source code. Just a spreadsheet with a bunch of hard-coded values. Amaury Séchet went a step further and explicitly told us not to trust anything in it. https://docs.google.com/spreadsheets/d/1rb2q5uaTl2JAm0Uhq1_7DmNg3Y3WHTWdhkd-ayorkNs/edit#gid=433230462
This is sloppy and unprofessional, and does not reflect well on Bitcoin ABC's competence and leadership quality.
8. Grasberg's simulation performance is inferior
So I did my own sims of Grasberg. I integrated Grasberg into my own simulation framework. A live web copy of my simulator is running at http://ml.toom.im:8051/. https://github.com/jtoomim/difficulty/blob/comparator/mining.py#L428 https://github.com/jtoomim/difficulty/tree/comparator
The Grasberg proposal has a few different variants in my simulator. The first one is labeled as `grasberg-288`, and emulates the behavior of Grasberg during the first 6.5 years while Grasberg targets a 11.25 minute block time. The second is `grasberg-neutral-288`, and reflects Grasberg's behavior after Grasberg has returned to a 10 minute block time target. I also included the `grasberg-nodrift-288` variant to simulate how Grasberg would perform if the drift correction code were ripped out entirely, as well as variants using different time constants for different resonsiveness/smoothness tradeoffs.
Grasberg's performance needs to be separately analyzed in two conditions:
Over the first 6.5 years of operation, while grasberg's performance is dominated by the 11.25 minute target block time
After Grasberg has neutralized the drift backlog against its genesis block-derived schedule
**Scenario 1** seems pretty straight-forward. Grasberg mines blocks slower than aserti3-2d and similar algorithms, and consequently gets worse confirmation times and mining fairness performance. But it's a bit worse than we expected: rather than performing 12.5% worse on confirmation times and mining fairness, `grasberg-288` performs about 19-25% worse.
When we switch to **scenario 2**, the algorithms start to look more similar, but a performance difference persists.
Once Grasberg has returned to a neutral 600 seconds per block, its performance is much improved. Across the board, grasberg has decent performance. However, it still does not perform as well as aserti3-2d.
I believe the main reason why Grasberg underperforms in these tests is because of Grasberg's choice of time constants. Bitcoin ABC did not follow my advice correctly on the choice of half-lives and time constants. (Ironically, the advice to use a 2-day half-life is the only thing that Bitcoin ABC credited me for, despite the fact that they did not follow the advice.) The source of this confusion appears to be the distinction between a *time constant* (or *relaxation time*), which would be tau in the natural exponential function `e^(x/tau)`, and a *half life*, which would be lambda in the base-2 exponential function `2^(x/lambda)`. I recommended the use of a 2-day *half-life*, and used that in my code; however, Bitcoin ABC ended up using a 288-block (2-day) *time constant*. The conversion from a time constant to half-life is done by multiplying by ln(2), so they ended up with a 199.6-block half-life. I informed Bitcoin ABC of this error on July 23, 2020, the day Grasberg was published. Amaury acknowledged the error, but has not fixed it. This appears to be because he did not simulate it in a scenario in which the difference manifests.
9. Grasberg is far more complex than ASERTI3-2D
Grasberg is far more complicated than it needs to be, in both implementation and design. This is readily apparent by comparing the aserti3-2d and Grasberg code side-by-side, but the complexity goes much deeper than that.
Mark Lundeberg (the co-inventor of ASERT) wrote a detailed review of the Grasberg implementation on July 24th. His review repeatedly raised concerns about Grasberg's complexity: https://www.reddit.com/r/btc/comments/hx4kgz/thoughts_on_grasberg_daa/
I find the detailed algorithm is unnecessarily complex for what it accomplishes, i.e., the same thing could be done much simpler. ... As alluded to above, I think the algorithm has way more complexity and computation than is needed. `deterministicExp2` is an approximation to 2^x - 1, for 0<=x<1, in fixed point math. It is a compound curve made up of 16 quadratic segments with discontinuities. It has a ~1 ppm accuracy (if I have translated it correctly). I find this algorithm is much more complicated than is needed for its accuracy level, but also I don't know why this accuracy level was selected in the first place: https://imgur.com/ZmkrtVk
10. Grasberg's complexity adds vulnerability to attacks
Fundamentally, Grasberg is neither absolute nor relative, neither ASERT nor RSERT; it's both simultaneously. Grasberg has a relative scheduling (RSERT) feedback loop for controlling mining difficulty over short time spans, as well as an absolute scheduling feedback loop that regulates the targeted block interval.
Not only does this dual loop system increase the amount of code Grasberg requires, but it also makes the behavior of grasberg more complex and adds vulnerabilities. While neither ASERT nor RSERT by themselves will oscillate, combining the two into a hybrid system does have oscillatory potential. The problem arises because the two parallel feedback loops can operate with different delays. The fast loop can complete its response to a perturbation in block interval while the slow loop is only getting started. This can leave a residual effect from the slow loop that can cause overshoot, ringing, and potentially oscillations. In the specific case of Grasberg, this oscillation is fortunately mitigated by the values that were chosen for some of the constants. Specifically, in the `computeTargetBlockTime()` function (which determines the slow loop's feedback and provides the drift correction), the constants `tau` and `X_CLIP` need to have high and low values, respectively, in order to prevent oscillation. If either one is changed enough, or if both variables are changed a moderate amount, then some instabilities can result. With larger changes, the oscillations become somewhat severe, as shown in the simulation run below with `tau = 600` and `X_CLIP = 2729822324`.
While the default values of `tau` and `X_CLIP` are sufficient to prevent any oscillation or bad behavior in natural mining scenarios, they can be successfully manipulated in adversarial scenarios. For example, there's a mining attack I call the fast forward attack which is exacerbated by Grasberg's dual feedback loop design.
The fast forward attack is an attack that all DAAs seem to be vulnerable to, albeit to different degrees. In the fast forward attack, a miner will create a secret chain which they mine for several days with 101% as much hashrate as the honest chain -- this part is somewhat similar to the reorg attacks that have been attempted against coins like Bitcoin Gold over recent years. But there's a twist: when the miner starts a fast-forward attack, they manipulate the timestamps of their blocks to be very close to the time at which they intend to publish the blocks. This fast forwarding of the timestamps causes the difficulty to drop, allowing for most of the blocks to be mined at a lower difficulty. Once the attacker gets to the same height that the honest chain will end at, the attacker will have used less hashrate to get the same number of blocks and coins. The attacker can then mine a few extra medium-to-high difficulty blocks until their chainwork exceeds the honest chain's. This gives them a few extra blocks overall. The amount of extra profit the attacker gets depends on the DAA and the exact mining strategy used. https://github.com/jtoomim/difficulty/blob/comparator/attacks.py#L19
My best attack against the cw-144 algorithm gets 209 blocks with as much work as 145 honest blocks, for 44% extra profit, whereas the aserti3-2d algorithm only allows 1138 blocks for the price of 816, or 39% -- that's a lower profit ratio for a larger (and harder) gamble. With Grasberg, the severity of the vulnerability depends on the `computeTargetBlockTime()` drift correction in the slow feedback loop. With it enabled, and after 6.5 years have elapsed and Grasberg is at equilibrium, **Grasberg can be fast-forward attacked for 52% extra profit.** But if `computeTargetBlockTime()` is bypassed (made to return 600 sec no matter what), then that extra vulnerability disappears, and Grasberg can only be attacked for 39%, which is same as aserti3-2d.
The fast-forward attack is not a particularly worrisome attack. Since it requires 101% of the hashrate and a multi-day reorg, whenever it is possible to perform it there are usually much more serious issues that we need to worry about. Also, the fast-forward attack is prevented by the 10 block finalization rule in ABC and BCHN. But it serves to illustrate the complexity of Grasberg's response, and suggests that there may be other potential attacks which are more practical and which are yet to be discovered.
11. Conclusion
In this article, I have shown Grasberg to be inferior in 10 different respects. In rough order of importance:
Grasberg will be 12.5% slower for 6.5 years.
Grasberg redefines the coin emission schedule, and sets Bitcoin ABC up as the central bank of BCH.
Grasberg changes the distribution of wealth.
Grasberg is widely opposed, especially in the BCH developer community, where 75% of DAA developers opposed it in a recent straw poll.
Grasberg's announcement was dishonest, ignored aserti3-2d, and appears to have been an attempt to steal credit from others for political gain.
Grasberg opens the door for BCH to be controlled via bribery and coersion.
Grasberg was not very well simulated by its creators.
Grasberg does not perform as well in simulations as the best-known proposals.
Grasberg's code is far more complex than a DAA needs to be.
Grasberg unnecessarily adds technical vulnerabilities to attack.
In the process of showing this, I have also shown disturbing patterns in Bitcoin ABC's behavior:
Bitcoin ABC has repeatedly misled the public. First, they withheld information about the 11.25 minute block times. Second, they falsely claimed that ASERT changes the coin emission policy, when that's actually what Grasberg does. Third, they claimed the nonexistence of other proposals.
Bitcoin ABC has repeatedly demonstrated technical incompetence and unpreparedness with Grasberg. This has been shown with the lack of solid simulation results, the potentially oscillatory design, the algorithm's complexity, and the algorithm's susceptibility to attack.
Bitcoin ABC has demonstrated a complete disregard for the positions and contributions of other Bitcoin Cash developers.
Bitcoin ABC appears to be taking a "You do you. We will do us" approach to software development. They appear to have developed Grasberg *in response to* aserti3-2d, and may be intentionally escalating to a chainsplit. https://old.reddit.com/r/btc/comments/hqbpri/why_do_we_need_to_change_daa_immediately_with/fxy5spf/?context=1
Bitcoin ABC is performing one of the gravest sins a developer can make: they are attempting to push through a change to Bitcoin's economic and monetary policy, and take on the role of a central bank.
Bitcoin ABC appears to be attempting to consolidate political power over Bitcoin Cash. Should they succeed in doing so, they will likely be able to leverage their power for money.
Amaury Séchet has done many good things for Bitcoin Cash in the past. In 2017, he and freetrader created Bitcoin Cash, and Amaury has worked hard for three years to maintain Bitcoin Cash's most popular full node, and to keep the code secured against attack. He helped us free ourselves from the control of Blockstream, after they transitioned Bitcoin's economy to a high-fee constrained-blocksize regime and attempted to push Bitcoin's users toward Blockstream's own product, Liquid. For his help with that, we will always be grateful to Amaury.
But over time, he has allowed his ego to grow unchecked, and has recently come to believe that he deserves more. In order to get what he thinks he deserves, he has attempted to consolidate control over Bitcoin Cash, and then leverage that control for income. First, he attempted to openly redistribute wealth to himself with the IFP, and his attempt was rejected by BCH's users. I respect the honesty and directness of the IFP, and do not fault Bitcoin ABC for it. Nonetheless, BCH sent a clear message to Bitcoin ABC in February: *Thou shalt not change the distribution of wealth.* Unfortunately, Bitcoin ABC seems to have not gotten that message, and is once again attempting to change the distribution of wealth. This time, there is the possibility of a path back into their own coffers, but no direct evidence of such a path. That said, there is direct evidence of dishonesty and subterfuge from Bitcoin ABC in other respects; and there's evidence that they're hiding *something*, and have not been forthright about their motivations for Grasberg.
12. What happens now
There is no coming back from this for Bitcoin ABC as an organization. This is the end-of-life for Bitcoin ABC.
Unfortunately, the severed head of a snake can still bite. As Bitcoin ABC collapses, they will still be able to do some harm to Bitcoin Cash in their death throes. It will likely be a few weeks before Amaury Séchet realizes that he has lost, and in the mean time, he will continue to struggle for dominance.
Bitcoin ABC has been playing a game of brinkmanship. They are using the fear of a chainsplit to try to convince everybody to conform to their leadership and their solution as a Schelling point. This fear of a chainsplit increases their chance of winning without a chainsplit: in the game of chicken, the person who swerves first loses. To counter this, we have two good options. We can remove their power to generate a meaningful chainsplit, and we can create a stronger Schelling point. If we do this properly, then we can change the game: the person who plays chicken on a motorcycle against a freight truck loses no matter what.
To remove Bitcoin ABC's ability to plausibly threaten a chainsplit, I recommend all Bitcoin Cash businesses, users, and investors take the following steps:
Every business, miner, and individual in the BCH ecosystem needs to stop running Bitcoin ABC software immediately, and switch to another full node implementation. Without users, Bitcoin ABC will have no bargaining power, and they will be unable to use the threat of a chainsplit to get their way. If you are running ABC, you need to switch immediately. It does not matter which node you choose; all are led by reasonable people. Depending on your business, you may want to choose:
Bitcoin Unlimited for high performance mining and block/transaction throughput https://www.bitcoinunlimited.info/download
BCHD for its advanced gRPC API and Neutrino support https://bchd.cash/
Flowee-the-Hub for its cutting-edge performance and API extensibility https://flowee.org/hub/
Bitcoin Cash Node (BCHN) for the thoroughness of their code review, or as a drop-in ABC replacement https://bitcoincashnode.org/
Bitcoin Verde for an integrated full node/block explorer platform https://bitcoinverde.org/documentation/
Knuth for a node-as-a-library interface from several programming languages http://kth.cash/
Revoke funding from Bitcoin ABC. If you are making monthly contributions to Bitcoin ABC, cancel them. Contribute financially to other projects if you can.
State publicly and clearly that you will not follow Bitcoin ABC or Grasberg in the event of a chainsplit, and will sell coins on any Grasberg chain. (This one is easy for me.)
If you can, offer jobs to Jason Cox, Fabien, and Antony Zegers. They're good engineers and good people who have been trying to make the best of a difficult situation.
If Amaury Séchet abandons Grasberg voluntarily, offer him a job as a developer, *but not as a manager*. Amaury is a good engineer who unfortunately was promoted until he was in a position of incompetence. Do not put him in a position where he will need to frequently review the work of others. Let him return to what he's good at: writing (but not reading) code. https://en.wikipedia.org/wiki/Peter_principle
To create a stronger Schelling point, what we need to do is simple:
If you prefer prefer aserti3-2d, say so, clearly and unambiguously. Do so as an individual or as a business. This is especially true for developers.
If we do these things, then we will make this transition as smooth as possible.
13. What comes next
On a more personal note, in 2018-2019 I was working on some benchmark projects and Xthinner development work based on Bitcoin ABC, but which I eventually abandoned because even simple changes got stalled in code review. Amaury seemed indifferent to my project, even when I demonstrated 3,000 tx/sec in my benchmark, and never engaged except to tell me that it needed more unit tests. A few months ago, as a way to ease my COVID blues, I decided to try resurrecting some of these projects for BCHN, and the difference in response was incredible. The BCHN devs were enthusiastic about the idea of stress test benchmarks. As soon as I published a merge request with draft code, they pored over it with detailed and simultaneous code review from several different devs on the team. Not only did they find problems in my code that I hadn't thought of, they *offered to fix them for me*, and then they made good on that offer. https://www.youtube.com/watch?v=j5UvgfWVnYg https://www.reddit.com/r/btc/comments/bgr143/xthinner_mainnet_compression_performance_stats/ https://www.reddit.com/r/btc/comments/hwoy8x/congrats_bch_the_daa_is_fixed_and_the_code/fz2kdn4/ https://reviews.bitcoinabc.org/D3025#71873 https://gitlab.com/jtoomim/bitcoin-cash-node/-/merge_requests/1
In mid-May, I realized that the deadline was coming soon if i wanted to get the DAA fix done for November, so I shelved the benchmarking/Xthinner stuff for the time being. Given the outpouring of help I had received earlier, as well as the fact that BCHN had always had a DAA improvement on their roadmap, it was an obvious decision to base my C++ aserti3-2d work on the BCHN codebase after I finished my Python3 simulation work. https://gitlab.com/jtoomim/bitcoin-cash-node/-/commits/xthinner https://news.bitcoin.com/bitcoin-cash-node-upgrade-plans/
I never once regretted it. After I published my proposal, developers started asking questions about the aserti3 design decisions in the BCHN slack, *including developers for other full nodes* like Tom Zander (Flowee) and Fernando Pelliccioni (Knuth). Tom Harding also chipped in in a few cases. On July 14th, after a couple of days of this, I was asked a question from freetrader about an hour after I went to sleep, asking me where and how they should do code review. Being asleep, I didn't respond, and freetrader and mtrycz didn't wait; they just did it on my repository, so I woke up to this. The next night was very different. Instead of comments like "Hey, there are problems X, Y, and Z, and you should add such-and-such unit tests," as I would expect at ABC, what I woke up to was that freetrader had forked my code, started fixing stuff, and started writing unit tests. Over the next few days, this continued back and forth, with each of us copying from each other's branches while the other slept, or occasionally submitting merge requests to each other. It was the kind of rapid, permissionless, chaotic development that open source development is supposed to be. It was productive, and it was *fun*. https://gitlab.com/jtoomim/bitcoin-cash-node/-/commit/fd92035c2e8d16360fb3e314b626bf52f2a2be67
I'm looking forward to getting aserti3-2d merged, fully tested, and ready for activation, because afterward I'll be able to work again on what I've always enjoyed most: scaling Bitcoin. If we can replicate the pattern we had with the aserti3-2d work on stuff like Xthinner and Blocktorrent, I think we'll be able to make some pretty rapid progress. I think I'm pretty good at doing research and coming up with new ideas and developing prototypes, and BCHN and freetrader have shown that they're pretty good at taking prototypes and turning them into well-specified production-ready code.
In a few months, we will be able to put this Grasberg drama behind us, and with a healthy development ecosystem, we will be able to can focus on building a better platform for p2p cash. We'll finally be able to fix all of the issues that Amaury stonewalled, like the 50 tx chain limit issue, building proper system benchmarks, parallelizing code, and adopting better block propagation technologies. Since 2017, Bitcoin ABC has put less work into code performance optimizations than any other full node that I've seen. Once Bitcoin ABC is no longer the network's bottleneck, we should be able to resume scaling BCH again, and turning Bitcoin Cash into something that will make Satoshi proud.
BCH upgrade proposal: Use ASERT as the new DAA
Abstract
The current BCH difficulty adjustment algorithm (DAA) allows for a switch miner to extract a large amount of profit from constant-hashrate miners. This strongly discourages constant-hashrate mining and encourages switch mining, and causes oscillations in difficulty and hashrate. In my test simulation, the stolen profit is on the order of 6% lost revenue by constant hashrate miners for the current DAA, but only about 0.35% to 0.65% for the EMA family of DAAs. I propose a specific type of EMA, called ASERT, for BCH to be activated in the November 2020 scheduled protocol upgrade, and provide extensive simulation results on this algorithm, as well as an untested sample C++ implementation. http://toom.im/files/da-asert.pdf https://gitlab.com/jtoomim/bitcoin-cash-node/-/commit/fd92035c2e8d16360fb3e314b626bf52f2a2be67#ee35cfc09ad35fa12b8d0a6280a950e4edde3d20_285_299
Why the DAA is a problem
The current BCH difficulty adjustment algorithm — known as `cw-144`, since the algorithm is based on chainwork — has a problem: it is prone to oscillation. This oscillation is the result of the current BCH DAA using a simple moving average, and has been described in detail in previous works. It was a known issue with `cw-144` even in October, 2017, before it was activated on mainnet.
(*If you're already familiar with this oscillation, such as if you watched my* *Feb 27, 2020 video**, feel free to skip this section and jump to the* ***Stable DAAs*** *heading.)* https://www.youtube.com/watch?v=Fd6GFpZjLxU https://github.com/zawy12/difficulty-algorithms/issues/48 https://medium.com/@Mengerian/dgenr8s-difficulty-adjustment-algorithm-explained-e77aa47eb281
There are a few reasons why this bug, and the incentive problem it engenders, are problematic:
It increases average transaction confirmation times by a factor of about 2x if 90% of the hashrate behaves in a rational manner;
It causes the chain to stall completely if 100% of the hashrate behaves in a rational manner;
It allows for some really nasty selfish mining attacks. Selfish mining attacks cause block reorgs, and reduce public confidence in the block chain, and make double-spend attacks more feasible;
It punishes miners for being loyal to BCH;
It punishes small solo miners who do not have the technological sophistication to automate hashrate switching strategies; and
It allows for malicious strategies in which a miner can intentionally amplify the oscillations and pro-switching incentives, either by switching their hashrate themselves in a pumped fashion, or by manipulating the timestamps of the blocks they mine.
In brief, the oscillation is due to `cw-144` using a simple moving average (SMA) over the last 144 blocks to estimate hashrate. When e.g. a high-hashrate block leaves the SMA window, it reduces the difficulty by the same magnitude as a new block entering the window would increase it. Blocks leaving the window therefore sharply change the difficulty in a direction that incentivizes miners to repeat the hashrate of the departing block. I call this effect "hashrate echoes." Notably, these echoes do not require any malicious miner behavior to occur. They are merely the result of miners following the direct short-term incentives that the DAA gives them.
These hashrate oscillations have a period of about 1.05 days, so they're not visible on most difficulty or hashrate charts (which use a 1-day sample interval), but they're clearly visible on this interactive chart. http://toom.im/bch_hashrate_new.html
These hashrate echoes first appeared in BCH shortly after the `cw-144` DAA was activated on November 2017. At that time, the oscillation amplitude was about 1 EH/s, with daily peak hashrate typically at 2x the daily minimum:
Due to changes in the behavior of a few large BCH mining pools, these echoes became significantly more severe around October 1st, 2019, prompting an article from BitMEX Research. The oscillation amplitude reached 11 EH/s, with daily peaks at 4x to 8x the daily minimum. Similar extremes were observed around January 20, 2020 and most of April, 2020. https://blog.bitmex.com/bitcoin-cashs-october-2019-hashrate-volatility-increase/
These oscillations in hashrate cause and are caused by changes in mining difficulty. These difficulty oscillations frequently exceed 10% in amplitude, causing dedicated miners to mine at a loss (compared to mining BTC) most of the time, with switch miners swooping in for around 2-6 hours per day to capture all of the profitable blocks.
These oscillations significantly worsen the user experience. The high-hashrate bursts are short; the hashrate droughts in between are long. Most transactions are issued during hashrate droughts, and the average transaction needs to wait much longer than the ideal 600 seconds for the normal Poisson block distribution. This effect can be seen in this interactive chart of confirmation times. Confirmation times have also gotten worse recently, averaging about 15 minutes since September, 2019, and peaking at 28 minutes in January, 2020. http://toom.im/bch_conftime_new.html
In Sep-Oct 2017, kyuupichan and a few other developers wrote the "difficulty" software suite to simulate these difficulty oscillations and hashrate switching with different DAAs. In Feb 2020, I extended this suite with a graphing browser-based UI (code; slow online version). These simulations confirmed that with the `cw-144` algorithm, these oscillations are produced without any malicious miner behavior. All that is necessary to generate them with `cw-144` is for miners to mine whichever coin (BCH or BTC) is the most profitable at any given moment.
https://github.com/kyuupichan/difficulty https://github.com/jtoomim/difficulty/tree/comparator http://ml.toom.im:8051/
The simulations also prove that the oscillations are specifically `cw-*`'s fault. The oscillations are present in all variants of the `cw` family when miners follow rational incentives, with larger and more frequent oscillations for `cw` variants with higher sensitivity (e.g. `cw-144` is worse than `cw-576`), but the oscillations are entirely absent from all other good DAA families, regardless of sensitivity.
The fact that it's possible for switch miners to earn 5-20% more than constant-HR miners is a bug. This bug is the result of the 1-day resonance and tendency for oscillation that the current DAA has. We can reduce the incentive for switch mining by about 90%-95% by switching to a different DAA, such as an EMA-family algorithm.
Stable DAAs
The instability in the simple-moving-average (SMA)-based `cw-144` algorithm is caused by the influence of a block on difficulty ending abruptly when the block leaves the 144-block window. This instability can be avoided by having the influence of a block fade slowly over time. For example, if the influence fades linearly over 144 blocks, the instability disappears. This is what the `lwma` (linearly-weighted moving average) algorithm does, as well as Tom Harding's related `wt` (weighted time) algorithm. Another option is to have the influence fade exponentially over time, with e.g. a 144-block time constant. This is what the EMA (exponential moving average) family of algorithms do. This family includes many variants, such as Jacob Eliosoff's `ema-1d` and `simpexp-1d`, Tom Harding's `wtema`, and Mark Lundeberg's `asert` algorithms.
Among these stable DAAs, we identified four major candidate algorithms: `asert`, `wtema`, `lwma`, and `wt`. All four algorithms performed well in my simulations and in zawy12's simulations. Below is a table of data on one such simulation run for all four candidate algorithms, plus `cw` for reference, at three different responsiveness settings corresponding to 1 day (as in the current DAA, `cw-144`), 2 days, or 4 days. The table includes the average confirmation time for transactions in the third column. That number should ideally be as low as possible. In the "Greedy" through "Steady" columns, the table shows the relative profitability of various mining strategies (compared to mining BTC). In the final column, the table shows the advantage to be gained by switching from the worst mining strategy to the best mining strategy. That number should ideally be as close to 0% as possible. https://github.com/zawy12/difficulty-algorithms/issues/62#issuecomment-646159947 https://github.com/zawy12/difficulty-algorithms/issues/62#issuecomment-645735330
Differences in hashrate regulation performance between these four options in simulations were minor. There seemed to often be a small edge for the EMA based algorithms (`wtema` and `asert`), but the advantages were small enough that we believed that the choice should be made on other concerns, like implementation complexity. Both `lwma` and `wt` require sampling every block header in their windows (e.g. 288 blocks) in order to calculate the next block's difficulty, whereas the EMA family generally only requires sampling two block headers, so our attention focused on the EMA family.
Our attention then focused on determining which EMA algorithm to use. The most interesting ones seemed to be `wtema` and `asert`.
WTEMA and ASERT
The two EMAs that seemed like the best candidates were `wtema` and `asert`. The `wtema` algorithm was created by dgenr8 (Tom Harding) on or before Jan 1st, 2018. It is an exceptionally simple integer-only EMA based on the following pseudocode: https://github.com/kyuupichan/difficulty/pull/30 https://github.com/kyuupichan/difficulty/pull/30/commits
next_target = prior_target / (IDEAL_BLOCK_TIME * alpha_recip)
next_target *= block_time + IDEAL_BLOCK_TIME * (alpha_recip - 1)
In abstract terms, it first estimates how much the block's "target" (the reciprocal of difficulty) should be changed in order to achieve 600 second blocks if the most recent block interval were perfectly representative of the hashrate. Then, it performs a weighted average between that value and the last block's target, using `1/alpha_recip` as the weighting coefficient. This makes `wtema` a recursively defined function, and a classic example of a simple first-order IIR filter, equivalent to an electronic RC filter. But don't be fooled by the simplicity: it performs just as well as a complex PID controller system, but with a fraction of the complexity. http://www.tsdconseil.fr/tutos/tuto-iir1-en.pdf https://github.com/zawy12/difficulty-algorithms/issues/20
The `asert` algorithm was developed mostly by Mark Lundeberg on or before February, 2020, though an equivalent formula was also independently discovered in 2018 by Jacob Eliosoff but without Mark Lundeberg's detailed analysis, and again in 2020 by Werner et. al at Imperial College London. `asert` is defined by the following formulae: http://toom.im/files/da-asert.pdf https://arxiv.org/pdf/2006.03044.pdf
target_timestamp = (height + 1 - activation_height)*IDEAL_BLOCK_TIME
next_target = target_at_activation * 2^((last_timestamp − target_timestamp)/tau)
*Note:* `^` *denotes floating point exponentiation, not integer XOR.*
`asert` stands for absolutely scheduled exponentially rising targets. In abstract terms, `asert` works by defining an ideal block schedule, and setting each block's difficulty exponentially, based on how far ahead of or behind that schedule its parent block was. For every `tau` seconds ahead of schedule a block is, the next block's difficulty goes up by a factor of 2. Reasonable values of `tau` can be from a few hours to a few weeks. Like `wtema`, `asert`'s code and math are simple, and only depend on two blocks. Unlike `wtema`, `asert` depends only on the most recent block and the genesis block or the fork block at which `asert` was activated. This avoids the recursive definition that `wtema` has. For more information on `asert`, I highly recommend Mark Lundeberg's succinct introductory paper. http://toom.im/files/da-asert.pdf
The two algorithms are functionally nearly identical when block intervals are reasonably close to normal, and only differ at the extremes.
The debate between the two algorithms seemed to center around four issues: https://github.com/zawy12/difficulty-algorithms/issues/62#issuecomment-646071315
Integer approximations — easier with `wtema` than `asert`, due to `asert`'s use of an explicit exponential function, but possible in both
Singularity and negative solvetimes — problematic with `wtema`, but not `asert` , as shown in the graph above (source: dgenr8). `wtema` likely needs rules restricting solvetimes to be greater than some value (e.g. > `-0.5 * 144 * 600 sec` for `wtema-144`) https://github.com/zawy12/difficulty-algorithms/issues/62#issuecomment-648958314
Mathematical and theoretical elegance — good with both, but better for `asert`
Accumulation of rounding/approximation error — modest in `wtema`, but absent in `asert` due to its absolute nature
The main issues seemed to be #1 and #2. The need for integer approximations of 2^x makes `asert` more complex. The need to protect against large negative solvetimes makes `wtema` more complex. Ultimately, we decided that it would be better to add a few lines of code to the difficulty calculation function itself than to add or change other consensus rules in order to forbid large negative solvetimes, so we settled on `asert`.
It is important to not include floating point code in Bitcoin consensus rules, since C and C++ are generally not IEEE 754-compliant, and different compilers and hardware can disagree on floating-point computations. So we need an integer approximation. We then began looking at different approximation implementations. https://stackoverflow.com/questions/5777484/how-to-check-if-c-compiler-uses-ieee-754-floating-point-standard
The method I chose relies uses the `2^x = 2 * 2^(x-1)` identity to shift the `2^x` approximation to the `0 <= x < 1` range. Once this has been done, `asert` becomes remarkably insensitive to any residual error, and even an approximation as simple as `2^x ~= 1 + x` works pretty well. But we could do much better than that, so we did. After testing a few approaches, I eventually choose to use Mark Lundeberg's cubic approximation of `2^x` for `0 <= x < 1`. This approximation maintains an error margin below 0.013%. https://twitter.com/MarkLundeberg/status/1191831127306031104
Here is an as-yet untested example C++ implementation of the `aserti3` algorithm, and here is a well-tested python3 implementation of `aserti3`. Performance mirrors that of true `asert` very closely. Testing of the C++ implementation should follow soon. https://gitlab.com/jtoomim/bitcoin-cash-node/-/commit/fd92035c2e8d16360fb3e314b626bf52f2a2be67#ee35cfc09ad35fa12b8d0a6280a950e4edde3d20_285_299 https://github.com/jtoomim/difficulty/blob/6787454d03d37107a5cc0c5fe35e2ec0c3aff7d0/mining.py#L373 https://github.com/zawy12/difficulty-algorithms/issues/62#issuecomment-647060200
The main competing approximation option was Jacob Eliosoff's `exp_int_approx()` implementation, which uses an algorithm grabbed from Stack Overflow to achieve arbitrary precision, but at the cost of the use of a `for` loop and more opaque math and code.
I chose a half-life of 2 days (`tau = 2*24*60*60`, equivalent to a responsiveness setting of `2*144/ln(2)` or `aserti3-415.5` due to a difference in exponential bases) because that seems within the range of values that perform near-optimally in simulation while also being easy for humans to understand and discuss — for every 2 days ahead of schedule a block's timestamp gets, difficulty doubles. I consider a half-life of 1 day (`aserti3-207.75`) to also be acceptable. I recommend against going lower than 207.75 or higher than 576. https://github.com/zawy12/difficulty-algorithms/issues/62#issuecomment-646866050 https://math.stackexchange.com/a/56064 https://github.com/zawy12/difficulty-algorithms/issues/62#issuecomment-646312380
Activation
Because the `asert` calculation depends on the fork block's height and difficulty, implementation is substantially simpler and easier if activation is on a predetermined block height, rather than at a predetermined median time past (MTP). If the latter approach is used, then post-fork the new `asert` code will need to search the blockchain for the first block whose MTP exceeded the fork time, and extract the height and difficulty from that block's header. In contrast, if a fork height is used, the `asert` algorithm just needs to search the blockchain for the difficulty at the known height — a considerably simpler operation. While an MTP-based activation of `asert` is possible, it is my belief that it will be simpler and better overall to do this activation based on block height instead.
Bitcoin ABC has a tradition of adding a poison pill to their code in advance to fork off obsolete versions of their software at a predetermined MTP. If a fork height is used to activate `asert`, then the poison pill activation and the `asert` activation will not coincide exactly. This will likely result in two technical hard forks in quick succession, rather than a single three-way fork. This is a minor disadvantage to using a fork height activation. But even if the poison pill does not coincide perfectly, it will still serve its purpose: It will discourage individuals from continuing to run obsolete software.
I believe that the (potentially permanently) simpler code of a height-based activation of `asert` outweighs the timing mismatch issue, so I used a height-based activation in my sample C++ implementation. I am okay with this being changed if a majority of the BCH development community disagrees. https://gitlab.com/jtoomim/bitcoin-cash-node/-/commit/fd92035c2e8d16360fb3e314b626bf52f2a2be67#00a11683af1b8f38bab65b15de6c73734d272438_74_76
Testnet
The standard 20-minute testnet rule applies: A block may be mined with difficulty 1 if its timestamp is at least 20 minutes ahead of the previous block's. Because `asert` difficulty adjustments are based on time rather than block height, the "timewarp" technique to reset difficulty on testnet (by jumping the timestamp forward repeatedly to lower the average chainwork over the last difficulty adjustment interval) does not work. Because of this, testnet difficulties will be slower to adjust after this change than before, and testnet users may be stuck with 20 minute blocks for a few days after an ASIC miner leaves testnet. This could be addressed by reducing the `tau` value on testnet, but the cost of this change would be that testnet would less accurately resemble mainnet. Developers who use testnet frequently should discuss which approach they prefer.
Attacks
We have looked at a few different types of attack against `aserti3`, and so far, with existing consensus rules and non-ABC/BCHN node policy, `aserti3` outperforms `cw-144` in almost all scenarios. There are only a few that seem concerning. All attacks that have been identified so far are small in magnitude (usually smaller than other known and existing attack vectors) and/or easily mitigated with small node policy changes.
The attack scenarios of significant concern are all ones that the current DAA protects against by using a median-of-three prefilter. The current DAA will use the median timestamp of the most recent three blocks as well as the median timestamp of the blocks 144, 145, and 146 blocks ago. This median-of-three prefilter effectively adds 1 block of delay to the DAA in most scenarios, which prevents difficulty from being exploited in a 2-block reorg attack, such as are used in most known selfish mining strategies. However, this extra block of delay also tends to cause oscillation issues in most DAAs that aren't already oscillatory, including EMAs, so zawy12 and I strongly advise against the use of median prefilters. This leaves two types of 2-block attacks as needing special attention. https://www.bitcoinabc.org/2017-11-01-DAA/ https://github.com/zawy12/difficulty-algorithms/issues/49
The first attack is selfish mining in the absence of oscillations. In this scenario, a selfish miner `A` can make a block `A1` with an artificially early timestamp — say, a solvetime of 1 second — which increases the difficulty of the next block `A2` by 0.24%. This can cause chain `A2` to have slightly more chainwork than `B2`, and can make nodes to prefer `A2` over `B2` even if `B2` was seen first. Bitcoin ABC and BCHN already have node policy rules that block this attack, but Bitcoin Unlimited and most other nodes do not. To address this issue, we recommend that all remaining nodes add hysteresis to the first-seen rule for blocks: nodes should only switch chains if the new chain's work exceeds the first's by some substantial threshold, like 10% or 50% of one block's chainwork. https://github.com/Bitcoin-ABC/bitcoin-abc/blob/master/src/validation.cpp#L2516 https://gitlab.com/bitcoin-cash-node/bitcoin-cash-node/-/blob/master/src/validation.cpp#L2699 https://github.com/BitcoinUnlimited/BitcoinUnlimited/blob/release/src/validation/validation.cpp#L1289
The second attack is the FTL (future time limit) attack. In this attack, a miner creates a block with the timestamp set to the FTL (currently 7200 seconds in the future according to most BCH nodes' policies) to lower the difficulty on a secret chain, then mines as many blocks as possible until the difficulty on that chain returns to normal. This attack can be slightly more effective for `asert` than for `cw` in some (but not all) circumstances because `asert`'s difficulty responds after a single block rather than two for `cw`. This effect is largely mitigated by the use of a slower responsiveness setting (`aserti3-2d` is about 1/4 as responsive as `cw-144`), but it is better addressed by simply lowering the FTL. https://github.com/Bitcoin-ABC/bitcoin-abc/blob/master/src/chain.h#L27
Lowering FTL appears to be a straightforward improvement. Many altcoins have lowered FTL with no observed negative consequences. In the modern world of NTP and GPS, keeping system clocks synchronized to within 0.1 sec is easy, so there's no reason to allow up to 7,200 sec errors without punishment. Even in humanity's interplanetary future, an FTL of 600 seconds will not cause problems for the Bitcoin protocol, because blocks in which the timestamp reflects the publication time will result in blocks being received with *old* timestamps on other planets, not future timestamps. In the far future, once bitcoin mining starts to be performed on vehicles traveling at relativistic speeds, we might have to rethink the FTL; but until then, I recommend lowering it to 600 seconds or less. Furthermore, whenever the FTL is reduced, the DEFAULT_MAX_TIME_ADJUSTMENT limit on how far peer time can differ from local time (the "peer time revert rule", as zawy12 calls it) must be adjusted accordingly. https://gitlab.com/bitcoin-cash-node/bitcoin-cash-node/-/blob/master/src/timedata.h#L13
This is not an exhaustive discussion of potential attacks, but merely an introduction to the known attacks of concern. Discussion of a few other attack scenarios can be found in zawy12's EMA for BCH (part 2) and Hash Attack Examples Github issue threads. https://github.com/zawy12/difficulty-algorithms/issues/62 https://github.com/zawy12/difficulty-algorithms/issues/18
Conclusion
The current DAA, `cw-144`, is causing a lot of problems for BCH. We should fix it. Switching to an EMA or LWMA algorithm would fix those problems. The `aserti3-2d` algorithm appears to be far superior to `cw-144` overall, and I believe it to be the best option we have.
A good DAA should have the following properties, in rough order of importance:
It should be stable, and not prone to oscillations;
It should keep confirmation times low;
It should keep incentives for miners to perform hashrate shenanigans low;
It should keep incentives for miners to perform timestamp manipulation shenanigans low;
It should keep incentives for miners to perform selfish mining shenanigans low;
Because of #3, #4, and #5, honest mining strategies with steady hashrate should get near-optimal income;
The chain should recover quickly after sudden changes in hashrate and/or exchange rate;
The average block interval should be kept close to the target (e.g. 600 seconds);
The algorithm should be mathematically simple and elegant;
The algorithm should be easily understood and analyzed;
The algorithm should be easy to implement elegantly and simply; and
The algorithm should have few or no edge and corner cases.
In this article, I have provided some evidence that `aserti3-2d` likely satisfies most or all of these requirements. I encourage the BCH developer community, and the BCH community as a whole, to search for scenarios in which `aserti3-2d` fails to satisfy these criteria, and to compare `aserti3-2d` to `cw-144` and any other candidate DAAs in those scenarios.
@jtoomim
