Hive is failing at failing: to scale gracefully over time.

avatar
(Edited)

1 hour


7 min


3 min

These videos all have the same theme; the theme that brought Graphene and DPOS into this world in the first place.

The scaling issue.

In 2017 this is all anyone could talk about.
$50 Bitcoin fees have that effect on people.

So the main idea here is that we shouldn't be fretting about scaling so much because everyone has been doing that since the beginning of the Internet. I still remember when everyone was saying 56k modems were the absolute limit and we'd never get faster Internet connections than that (5-10 KB per second). Today I can download hundreds of megabytes per second. If the Internet gets much faster it will no longer be the bottleneck and I'll have to get a faster computer just to keep up.


Transferring pictures online used to be an unheard of (almost offensive) concept. Then it was audio, then video, then high-definition video, then streaming high-definition video.

assciporn2.png
So hot right now.

As soon as more space becomes available for scaling, be it the Internet, Bitcoin, or whatever else, those resources will immediately be consumed by the engineers and innovators that know what to do with it.

So how is Hive failing?

When's the last time you heard about this network having a scaling issue? When's the last time full nodes complained about being overburdened with requests or the last time we had to increase resource credit costs to limit how much users could post to the blockchain? I'll wait.

The last time I remember having a scaling issue was when I first got here in 2017. During high traffic times the witnesses were limiting the bandwidth of users and you couldn't transact on the blockchain unless you had enough powered up stake to do so. This was before the resource credit system even went into effect, so we haven't even had a chance to test the limits of our scaling ability with the cool new system we (Steemit) invented.

Isn't that a good thing?

No...

Quoting myself:

those resources will immediately be consumed by the engineers and innovators that know what to do with it.

This is the problem; we DON'T know what to do with our resources. No one is running out of resource credits. Blocks are not filling up. Full nodes are not being overloaded with requests.

We don't know our limits because we aren't testing them.

I remember when the RC system went into effect. Do you? We all had negative resource credits and no one could transact on the blockchain because the hardfork retroactively acted like we had been using the system from the genesis block.

More importantly, how we solved that clusterfuck was most alarming. We basically cut the cost of transacting on the blockchain by a factor of 10 so that users with less RCs would still be able to post to the blockchain. This leads me to believe that the RC system is totally broken (or at least extremely uncalibrated) and we don't even know it.

Imagine implementing RC pools. This would basically allow all those whales to start using their credits, whereas today those credits are either being unused or simply used to purchase account tokens to be redeemed at a later date. Allowing whales to use their RCs for real is going to dump massive inflation onto the system, and many consequences could develop:

  1. The RC costs to transact could go sky high because whales are actually spending them. There's a good chance we'd have to x10 the cost of everything to reverse what we did back in the day. This would give RCs value and whales would be selling theirs for a profit.
  2. Sooner or later, blocks will start filling up and we'll have to discuss raising the blocksize limit. This is exactly why Bitcoin forked to Bitcoin Cash. Again, if blocks start filling up we have to raise RC costs and cut the little guys out of the equation unless they have access to an RC pool.
  3. Second layer and off-chain solutions start being developed to migrate data that doesn't necessarily need to be on chain in the first place. The more RCs are worth the more financially incentivized we as a network become to remove data from the chain if it doesn't absolutely need to be there.

None of these things are happening because no one is utilizing this chain to its full potential.

It worries me that we are going to hit the next mega-bullrun and be wholly unprepared just like we were last time along with the rest of the cryptosphere. We'll all hit that scaling wall, servers will start melting, and the space will be declared dead again as we trudge into the next bear market. Perhaps that's just the way of things.

https://peakd.com/hive-139531/@blocktrades/misconceptions-about-2nd-layer-apps-part-1

I find the timing of this article by @blocktrades of particular relevance. Essentially he is describing a dapplication that exists directly on the blockchain. We need more of those if we are going to test our limits. We should all thank Spliterlands for migrating over here and essentially posting operations on what seems like every block. Nice work!

facemaskshistoryofmasqueradecaliforniacovid19S.jpeg

Masquerade

We have a lot of "dapps" here that claim to be decentralized, but are they really? Booting up a centralized server that you control and then connecting it to Hive with a thin strand does not make it decentralized. Just because you're piggybacking off of Hive's account security and currency doesn't mean you've created any kind of trustless, permissionless, or borderless system.

Is this a bad thing? Not necessarily. It's probably the only reason why this network hasn't melted down due to lack of scaling yet. If everything was on chain... 65KB every 3 seconds is not enough space to run it all. That's a fact.

Hive is not good at scaling just because it scales better than Bitcoin and Ethereum.

The implications of distributed ledger technology are that it's an inefficient database who's data is copied and verified thousands of times across the globe to ensure consensus and trust in the network. Hive is not different.

Imagine we have 10k daily users (we do). That's fine. Now imagine 100k, 1M, and 10M. Did you know Myspace still has over 10M users? Yeah, we can't even scale as large as laughable Myspace. Think about how difficult it would be to scale up huge servers when those huge servers have to be copied 1000 times across the world by every other person running a node. Truly, we have to carefully pick and choose our level of decentralization to remain relevant in the space.

Conclusion

It seems like a silly thing to complain about: failing to fail. However, some of the greatest lessons are taught in the wake of failure. Better to learn those lessons sooner rather than in the middle of a mega-bullrun when shit is really hitting the fan. There's not much I can do about it except continue on trying to develop actual decentralized dapps with provable ownership and the inevitable inefficiency that comes along with it. If anything, it would be nice if more people were aware of this "non-issue". In the end we'll likely just have to wait around until we have enough devs to really test the limits of this place.



0
0
0.000
12 comments
avatar

I remember reading a post or comment by someone (can't remember who) long ago, something about the possibility of only the changes (in new blocks I guess) being served to browsers when someone refreshes a page, and this would save a huge amount of data. Maybe that might help.

Other than that, what alternatives are there to distributed ledger? I read recently about a newish chain (maybe Band protocol) which does things in a different way.

How can we get Devs to look at this before saturation point?

0
0
0.000
avatar

When talking about getting people on this network to agree... well lets just say that you can't herd cats. We couldn't even come together to stop a hostile takeover even though we had the votes to do so. Realistically devs are going to do what devs want to do here. That's the really fun part about this place: answering to no one; being your own boss; yada yada yada.

We really just need to keep building and be patient. When we hit that scaling wall it will force itself into the limelight and become the main focus once again. I believe the system we've built might be good enough. When blocks start filling up users will have to pick and choose which operations are worthy of their precious RCs, and that's fine.

0
0
0.000
avatar

Getting large groups of people to agree on anything anywhere is very difficult. In smaller groups however it's much easier. It seems groups of eight is the optimum.

I will attempt to explain how multiple groups of eight can help a very large group come to agreement.

With the Matrix-8 consensus system (which I think we very briefly discussed in discord months ago) people are grouped in 8-PAC's (8Ps) (8 People in Active Communication) and a consensus (an actionable accord - AA) can be achieved often quite easily. Then a Delegate is elected.

Next a delegates meeting of 8 delegates takes place, with the remaining 56 people listening in, thereby hearing the other 7 AAs from 7 other 8Ps. (this is an example for a group of 64. With larger groups there would be several delegates meetings happening simultaneously).

If a delegate misrepresents (does not correctly speak the AA agreed upon) he can be immediately un-delegated and a new delegate elected who takes his place in the delegates meeting.

So, as the remaining 7 peeps in an 8P listen in and hear the other AAs, they can amend their AA to align with one of the other AAs, or make a new one, then an elected secretary from the 8P sends text to the delegate to inform him of change which he then conveys in the delegates 8P meeting. This continues, in real time until the delegates 8P reaches a concentrated AA.

This is infinitely scalable so a meeting of say 10000 peeps in groups of 8 can reach an AA in a couple of hours I think. Or ten million still in less than half a day.

There's a lot more to it, with Anonymity and Trusted Reputation being key components of the consensus mechanism. I made a 2 minute slideshow (not yet complete) a few days ago to show this visually which helps make it easier to understand:

▶️ Watch on 3Speak

A draft presentation of The Matrix-8 Solution consensus ... https://peakd.com/hive-181335/@atma.love/gubkfeff

Hit pause between slides to give you time to read each slide fully.

Posts about Trusted Reputation, Anonymity, and more about The Matrix-8 Solution can be found in the New Age DApps community here https://peakd.com/c/hive-153630/created Follow the comment threads in the recent posts too.

The Matrix-8 Solution could bring hundreds/thousands of new Hive users every time a new Matrix-8 sub-platform is deployed. Once a critical mass is reached there'll be a fond farewell to corrupt (all) governments! This post is def worth a read on that point https://peakd.com/hive-153630/@atma.love/to-hive-or-not-to-hive-2-aug-20

I invite you (urge you) to research and ask questions @edicted. Make it a business if you wish with Matrix-8 sub-platforms for businesses, communities etc. But the master platform (to which world population can connect) should be owned by no-one. There is a model for this. I can connect you with the designer @matrix-8 if you are interested to discuss further in a voice call, and/or with me, but he knows the system better.

I have already offered 5-10k investment to help develop a DApp once we have enough serious interest (see my post in the community).

Ok, enough for now. I didn't mean to hijack your post, but this is very relevant imo.

Namaste

0
0
0.000
avatar
(Edited)

Incidentally, I note this post has over 100 votes yet only 1 comment (well, two now😀). I propose a change so half the amount of voters share of voting rewards are sent to @null unless the voter has also made a comment (of at least X no. of characters. Not just for your posts though @edicted 😂. What do you think?

0
0
0.000
avatar

Unfortunately rules like that can't be enforced on a decentralized network. The more rules we make the easier it is for devs to game the system with bots. Something like would almost certainly incentivize pages and pages of spam to be posted to the blockchain. A dev like me would come along and create a script that anyone could use to make sure their votes weren't getting dusted.

0
0
0.000
avatar

Well, not enforced of course, it would need to be by consensus. And I think there could be a way round what you say, like for instance spam comments to get round losing half the rewards, with a certain number of downvotes from a trusted (an elected by consensus) downvote community could send all the vote reward to null. I think there is a solution for everything, but then I'm an eternal optimist (yet also realistic). Anyway, that thought can stay on the back burner for now as we don't (yet) have a good enough consensus mechanism.

Thanks for responding.

0
0
0.000
avatar

Sure but then this "trusted" entity centralizes the chain a bit and becomes the next attack vector. This idea also implies that users shouldn't be able to distribute inflation at full strength unless they are socially engaging with the content creators, which is a whole new argument.

What if I create an app that doesn't fit that narrative? What if I create an app that depends on upvotes but there is no social mechanic? You'd be inadvertently cutting me down at the knees because you figured the rules of consensus should solely apply to the social media aspect of the platform.

When I first got here in 2017 I had dozens of ideas about how to "fix" all the "problems" with this place, but as time went on and I learned more about the platform (especially programming stuff) I found that many of my ideas were ridiculous and were either impossible to implement (due to no KYC and the bot element) or attempting to solve a problem that didn't really exist (or that I was over-exaggerating. These days I tend to lean toward freedom to do whatever we want without over-complicating the system with a bunch of rules that may or may not work.

A lot of the ideas that people have for this place are very good ones, but they need to be tested on a second-layer token first to make sure they actually work and are worth the hardfork it would take to bring them to life.

0
0
0.000
avatar

Yes, absolutely it would need to be tested in a tokenized community first. Definitely the Matrix-8 Solution would have to be. For sure it wouldn't get sufficient VP to put it in place anyway until it has proved it's worth, and even then it will likely take a large number of users using it in many communities before there's enough VP to get it voted in to replace DPOS. It might however be that another or new Blockchain (without the whale power likely resisting it), seeing the success of the consensus mechanism in Hive communities and outside organisations/businesses who adopt a platform, will pick it up before Hive. I'm gunning for Hive though.

0
0
0.000
avatar

...account tokens to be redeemed at a later date.

It should be obvious that the ability to create accounts has value. Shouldn’t one of the Hard Forks we’re doing make it possible to transfer account creation tokens? Shouldn’t there be a market for something that has value? You’ve been able to create an account at the cost of 3 Hive for quite a while, an arbitrary fixed price. Shouldn’t there be a price discovery mechanism? Right now, 3 Hive might well be an artificially high price, but without a market, we have no way of really knowing.

0
0
0.000
avatar

The purpose of charging 3 Hive for an account is purely defensive in nature. If they were free or very low cost we could get attacked by bad actors essentially "domain name squatting" or just attempting to overburden the network.

I will agree that unused account tokens certainly are a fungible asset that could be traded. We could easily pair them to HBD on the internal market. Once we have resource pools we could get rid of the 3 Hive peg and create them entirely from resource credits. To me this would make more sense in the long run, as capping their value in a free market seems odd at best. That scenario would still allow the network to increase and decrease the value of account tokens based on the RC cost to create them... implying that RCs also have a definitive value based on the free market.

0
0
0.000
avatar

Lol Faling to scale can be temporary,scaling to fail is usually a death sentence! I’d be interested to see how this chain holds up if we get another one of those 2017 run ups and people looking to get their hands on any bit of crypto and we get that 250k daily active users level where it just goes nuts not only on the price but the chain activity too

Also now with all the forks going who knows which one people will pick, steem still got that good old branding

0
0
0.000
avatar

Not quite sure what you mean by scaling to fail. If I had to guess it would be putting too much overhead cost into scaling when the volume and profits aren't there yet.

https://peakd.com/bubble/@edicted/ridiculous-dot-com-bubble-comparison

WebVan
Online grocery shopping.

By creating a mammoth, $1 billion infrastructure of high-tech warehouses across the U.S., WebVan squandered the $375 million it raised at IPO on growth that its revenue simply could not justify. The other major problem facing WebVan was the already thin profit margins of the grocery business.


I'm not asking for any resources to be diverted into scaling this network up.
I'm asking that the devs actually use what's been given to us.
Big difference.

0
0
0.000