Cloud computing death countdown, the future belongs to distributed peer-to-peer network

Even from the perspective of the Internet, P2P is not a new concept. P2P is the foundation of the overall architecture of the Internet. The most basic protocol of the Internet, TCP/IP, does not have the concept of a client and a server. All devices are equal parts of communication. Ten years ago, all systems on the Internet had both server and client functions.

When cloud computing was hot, Viktor Charypar, technical director of British digital consulting firm Red Badger, wrote on VentureBeat that cloud services will usher in the end. And pointed out that peer-to-peer network will be the future direction of development. He said that the cloud will come to an end. This is a bold conclusion, and it may sound a bit crazy. But please bear with me, let me go on.

Cloud computing death countdown, the future belongs to distributed peer-to-peer network

There has always been a traditional view that applications running servers, whether they are web applications or mobile applications, will be in the cloud in the future. Amazon, Google, and Microsoft have added tools to their cloud services to make software services running in them easier and more convenient, so hosting code on AWS, GCP, or Azure is the best you can get. Things - it's convenient, cheap, easy to automate, and you have the flexibility to control the scale...

So why should I predict that this will all end? There are several reasons for this:

First, it cannot meet long-term expansion requirements.

Building a scalable, reliable, and highly available web application is difficult, even in the cloud. If you do a good job and make your app a great success, then the sheer size will drain your money and energy. Even if your business is very successful, you will eventually reach the limit of cloud computing: the computing speed and storage capacity of computers are growing faster than the bandwidth of the network.

Ignore the network-neutral argument, which may not be a problem for most people (except Netflix and Amazon), but it will happen soon. As our video quality ranges from HD to 4K to 8K, the amount of data we are in need is growing dramatically, and VR data sets will soon appear.

This is a problem, mainly because of the way we organize our networks. There are many users who want to get content and use programs, and only a relatively small number of servers own these programs and content. For example, when I saw a funny photo on Slack, I wanted to share it with 20 people sitting next to me, but they all had to download it from the server hosting the service. The server needs to send this photo 20 times.

As servers move to the cloud, such as Amazon or Google's data center Amazon or Google's computers, networks close to these places require incredible throughput to handle all of this data. In addition, you must have a large number of hard drives to store everyone and CPU data, and then transfer the data to everyone you want through the network. With the rise of streaming services, the situation has gotten worse.

All of these activities require a lot of energy and cooling, making the entire system inefficient, expensive, and environmentally unfriendly.

Second, it is concentrated and fragile.

Another issue that centrally stores our data and programs is usability and durability. What if Amazon's data center is hit by an asteroid or destroyed by a tornado? Or, what if it loses power for a while? Data stored on its machines cannot now be suspended or even permanently lost.

We usually alleviate this problem by storing the data in multiple locations, but that only means more data centers. This can greatly reduce the risk of accidental losses, but what about the data you care about very much? Your wedding video, photos of your child's growth, or important sources of public information, such as Wikipedia. All of this information is now stored in the cloud - on sites like Facebook, Google Drive, iCloud or Dropbox. What happens to the data when these services stop operating or lose money? Even if they don't develop this step, it limits the way you access your data. You have to use their services. When you share it with your friends, they must also pass the service.

Third, it requires trust, but it cannot provide protection.

With cloud services, you have to make friends believe that the data they get is what you send, and that this data is passed through trusted middlemen. In most cases, this can be benign and acceptable, but the websites and networks we use must be registered to operate legally, and the regulators have the power to force them to do a lot. In most cases, this is a good thing that can be used to help solve crimes or remove illegal content from the web, but there are many cases where this power is abused.

Just a few weeks ago, the Spanish government did everything it could to stop the independent referendum in Catalonia. The measures taken included blocking the information website and telling people where to vote.

Fourth, it makes our data more vulnerable to attack.

The real scary aspect of a highly concentrated Internet is the concentration of personal data. The big companies that serve us have a lot of data -- it contains enough information to predict what you are going to buy, who you will vote for, you might buy a house, and even how many children you might have. This information is enough to handle a credit card, a loan, or even a house in your name.

Moreover, you may agree. After all, if you choose their services, you can only trust them. But this is not what you need to worry about. What you need to worry about is other people. Earlier this year, credit reporting agency Equifax lost data on 140 million customers, one of the biggest data breaches in history. These data are now public. We can think of this as a ten-year event. If we are more careful, this situation can be avoided, but it is increasingly obvious that data leakage like this is difficult to avoid completely. And once it appears, it is too dangerous to be tolerated. The only way to really stop this type of incident from happening is to not collect such large amounts of data from the start.

So, what will replace the cloud?

The Internet, which is primarily supported by client-server protocols (such as HTTP), and the security based on trust to central authorities (such as TLS) are flawed and can cause problems that are largely difficult or impossible to resolve. Now is the time to look for something better - a model framework where no one else can completely store your personal data, large media files spread throughout the network, the entire system is completely peer-to-peer and serverless (I don't mean that kind of cloud) In the sense of "no server" in the sense of hosting, I mean there is no real server).

I have read a lot of literature in this field, and I am very convinced that peer-to-peer is the inevitable development direction of our future. Peer-to-peer network technology is a building block that replaces the network we know with protocols and policies, solving most of the problems I mentioned above. The goal is a fully distributed, permanently redundant data store where each participating user is storing a copy of some of the available data.

If you've heard of BitTorrent, then the following should sound familiar. On BitTorrent, network users can break big data files into smaller chunks or fragments (each with a unique ID) without any central authority authorization. To download a file, you only need a "magic" number, which is a hash, that is, the fingerprint of the content. Then, your BitTorrent client will find the users who own the file fragments by "content fingerprinting" and download the file fragments from them until you have all the clips.

An interesting point is how to match users. BitTorrent uses a protocol called Kademlia. In Kademlia, each peer on the network has a unique ID number that is the same length as a unique block ID. It stores a block with a specific ID on a node whose ID is "closest" to the ID of the block. The random ID of the block and network peers should be fairly consistent across the entire network. However, the block ID does not need to be randomly chosen, but instead uses a cryptographic hash - this is the only fingerprint of the block's own content, which is beneficial. Make sure these blocks are addressable. This also makes it easier to verify the contents of the block (by recalculating and comparing the fingerprints) and to ensure that the user cannot download other data than the original data.

Another interesting feature is that by embedding the ID of one block into the contents of another block, you can connect the two together in a way that will not be tampered with. If the content of the link block changes, its ID will change and the link will be destroyed. If the embedded link is modified, the ID of the containing block will also change.

This mechanism of embedding the ID of one block into another block makes it possible to create such a blockchain (such as bitcoin and other cryptocurrency-driven blockchains), or even more complex structures, usually Known as Directed Acyclic Graphs (DAG). (After Ralph Merkle invented this link, it's often called a "Merkle link." So if you hear someone talking about Merkel DAGs, you probably know what they are saying.) A common example of Merkle DAG is Git repository. Git saves the commit history and all directories and files in a huge Merkle DAG.

This leads to another interesting feature of distributed storage based on content addressing: it is immutable. Content cannot be changed. Instead, the new revision is stored next to the existing revision. Blocks that have not changed between revisions are reused because they have the same ID by definition. This also means that the same files cannot be copied in such a storage system and converted into efficient storage. So on this new network, every unique and funny picture exists only once (although there are multiple copies in the entire group).

Protocols like Kademlia, the Merkle chain, and Merkle DAG give us the tools to model file levels and revise timelines and share them across a large P2P network. There are already some protocols that use these technologies to build distributed storage that meets our needs. What looks promising is IPFS.

Name and sharing issues

Ok, with these techniques, we can solve some of the problems I raised at the beginning: we get distributed, highly redundant storage on devices connected to the network, which can record the history of files and when needed All versions are retained. This (almost) addresses availability, capacity, persistence, and content verification issues. It also solves the bandwidth problem—because it uses peer-to-peer data, so there's no way the server can't handle it.

We also need a scalable computing resource, but that's not difficult: everyone's laptops and phones are now more powerful than most applications (including fairly complex machine learning calculations), and computing is usually It is scalable. Therefore, as long as we can make every device do the necessary work for the user, there will be no big problems.

So now the funny pictures I see on Slack can come from a colleague sitting next to me, not from Slack's server (not crossing any "ocean" in the process). However, in order to post a funny photo, I need to update a channel (that is, the channel will no longer look like it was before I sent the message, it will change). This sounds quite simple is the hardest part of the whole system.

The most difficult part: real-time updates

The concept of an entity changes over time, which is actually a human-only idea that gives the world a sense of order and stability in our minds. We can also think of such an entity as an identity or name that will present a range of different values ​​over time (this is static and immutable). . Simulating information in a computer is a more natural way to produce more natural results. If I tell you something, I can't change what I said to you anymore, and there is no way for you to forget it. For example, the president of the United States does not change over time; it is simply replaced by other facts (people) of the same nature (identity and name). In the Git example, a ref (branch or tag) can point to a different commit (holding an ID and a value) at different points in time and submit a commit to replace the currently held value. The Slack channel also represents an identity that has grown in value over time.

The real problem is that we are not the only ones who have access. Many people try to post messages and change channels, sometimes at the same time, and someone needs to decide what the results should be.

In a centralized system, almost all current web applications have a central entity that determines the result and serializes the event. However, in a distributed system, everyone is equal, so there needs to be a mechanism to ensure consensus on the network.

For a truly distributed network, the most difficult problem to solve this problem is all the applications we are using today. Not only does it affect concurrent updates, but it also affects other updates that require “real-time” updates—the “single source of truth” is changing over time. This problem is especially difficult for databases, and it also affects other critical services, such as DNS. Registering a person's name for a particular block ID or series of IDs in a decentralized manner means that each participant needs to agree that an existing name has a specific meaning, otherwise two different users can see two Different files are under the same name. Content-based addressing solves the problem of the machine (remember that a name can only point to a specific match), not a human problem.

There are some major strategies for dealing with distributed consensus. One of the problems is to choose a relatively small “group” manager whose mechanism is to choose a “leader” that determines the truth (if you are interested, look at the Paxos and Raft protocols). All changes go through these managers. This is essentially a centralized system that compensates for the loss of central decision entities or interruptions ("partitions") in the network.

Another approach is a verification-based system like the Bitcoin blockchain, in which an update is written by letting the user solve a "difficult problem" (for example, adding a valid block to the Merkle chain). To reach a consensus. This "difficult problem" is difficult to solve, but it is easy to find out. If the conflict still exists, some additional rules are needed to solve it. Several other distributed blockchains use a verification-based consensus while reducing the energy requirements needed to understand the problem. If you are interested, you can read the relevant evidence in the BitFury white paper.

Another approach to specific problems is around CRDT, a conflict-free type of replicated data that will not suffer from consensus problems in certain situations. The simplest example is an incremental counter. If all updates are just "add one", as long as we ensure that each update is only applied once, the order is not important and the result will be the same.

There seems to be no clear answer to this question, and there may never be an answer, but there are many smart people who are working hard to solve this problem, and there are already many interesting solutions to choose from. You can only weigh it. This trade-off is usually reflected in the size of the group you "target" and the one that you pick from the consensus you are willing to give - usability or consistency (or, technically, network partitioning, but In a highly distributed system, this seems hard to avoid, just like the system we are discussing). Most applications seem to support usability rather than immediate consistency—as long as the state is consistent within a reasonable amount of time.

Privacy issues in public file networks

One obvious problem that needs to be addressed is privacy. That is, how to store content in a distributed cluster without being exposed? If it can hide things, then content address storage is a good choice, because in order to find something, you need to know the hash of its content. So essentially we have three levels of privacy: public, hidden and private. The answer to the third question seems to be in cryptography - strong encryption of stored content and the need for "external" shared keys (such as paper sharing, NFC device transmission or scanning QR codes, etc.).

Relying on encryption technology may sound risky at first (after all, hackers will always find vulnerabilities), but it's actually not worse than what we do today. In fact, this may be better in practice. Businesses and governments often store sensitive data (including the individuals involved in the data) in ways that are not shared with the public. Instead, only a small number of employees employed by the organization that owns the data can access it, and at least the password is protected. Normally, if you have access to the system that stores the data, you can have all of this data.

But if we turn to storing private data in a way that is essentially open, then we have to protect it (using strong encryption), which is bad for anyone who gains access. . This idea is the same as the original intention of the developer's open source code for security-related software, so that anyone can view and discover the problem. Knowing how the security system works should not help you break it.

An interesting feature of this control of access is that once you grant someone access to certain data, they will always modify the data. Of course, you can change the encryption key at any time. This is no worse than what we are today, although it may not be obvious: anyone can copy it if they have access to some data.

An interesting challenge in this area is to build a good system that can authenticate and share private data among a group of people who need to change over time. For example, there is a set of collaborators in a private Git repository. This can be achieved with a combination of private key ciphers and spin keys, but giving users a smooth experience can be a challenge.

From cloud to fog

Although we have to solve some problems, we will migrate from cloud computing, which will be a very exciting future. First, on the technical side, we should get quite a bit of improvement from a peer-to-peer network. Content-addressable storage can provide cryptographic verification of the content itself with untrusted authorization and permanent hosting (as long as someone is interested in it), we should be able to see a significant increase in speed, even It's on the edge of the developing world far from the data center (even on another planet).

At some point, even the data center can become history. Consumer devices have become so powerful, ubiquitous, computing power and storage space are almost everywhere.

For companies running web applications, this change will save huge costs. Companies will also be able to reduce their focus on downtime risks and focus more on increasing customer value for everyone. We still need cloud hosting servers, but they are just one of many similar servers. We can also see more diverse applications in which not all applications are the same—in the same application, there are consumer-oriented and back-end, and the difference is just access. The permissions are different.

Another great benefit for businesses and customers is the processing of customer data. When there is no longer a need to centrally store large amounts of customer information, the risk of losing this data is reduced. Leaders in the software engineering community (such as Erlang founder Joe Armstrong, whose conversation is worthy of attention) have long believed that on the Internet, the design of data sent by customers to corporate programs is a degradation, and companies should be Send programs to enable them to execute private data that is not shared directly. Such a model seems to be more secure and does not in any way prevent companies from collecting the useful user metrics they need.

Moreover, there is currently no way to block mixed service models that are opaque and retain private data.

This type of application architecture seems to be a more natural way to provide large-scale computing and software services, but also closer to the idea of ​​open information exchange, anyone can easily forward content to others, and control can The content that is published and accessed, not controlled by the private entity that owns the server.

Transparent Led Film Screen P8

We are an enterprise with the whole industry chain of LED film screen display platform, which realizes the research and development of transparent film materials, independent patch production, assembly, sales and service. Products are mainly used in large glass curtain walls, city buildings, large shopping malls, glass guardrails, glass windows, shops along the street, automobile 4S stores, jewelry brand stores, brand clothing chain stores, unmanned supermarkets, indoor hanging and other fields.

Transparent Led Film Screen P8,Small Transparent Led Screen,Transparent Led Film,Transparent Led Wall

Guangdong Rayee Optoelectronic Technology Co.,Ltd. , https://www.rayeeled.com

Posted on