Internet was taken down by one small typo at Amazon's web services, company says

Amazon accidentally took down large swathes of the internet with just one typo.

A worker retrieves goods from shelves at Amazon's warehouse on December 5, 2014 in Hemel Hempstead, England Peter Macdiarmid/Getty Images

A worker retrieves goods from shelves at Amazon's warehouse on December 5, 2014 in Hemel Hempstead, England Peter Macdiarmid/Getty Images

This week, many of the world's biggest websites and services stopped working because of a problem with Amazon Web Services, the platform that the retailer provides to power people's websites. One of its sites went offline because of the issue – and since thousands of websites rely on it, including many of the world's biggest, they immediately went competely offline or stopped working.

Websites like Quora, Trello and some of the world's biggest news sites went offline or stopped working properly when the issue happened. It even emerged that people's houses broke down – internet-enabled ovens, lights and front gates stopped working as a result of the outage.

Now it has emerged that all of those problems were the result of just one small typo on a set of servers.

The team at Amazon's Simple Storage Service were working to remove a small set of servers from its system, according to a technical report into the incident. As it did so, someone entered the wrong command and removed "a larger set of servers was removed than intended".

One of the two servers that were affected was one that manages the "metadata and location information of all S3 objects" in the Virginia data centre, according to the post. That meant that many of the core processes broke down and the server centre could no longer be used.

To fix that problem, Amazon had to fully restart all of the affected systems. But that was a huge operation – Amazon said it hadn't done one for "many years – and it took even longer than expected, meaning the problem couldn't be fixed quickly.

The company says it has now added a range of fixes to stop the issue happening again. It has modified the tools that deal with such problems to cope more efficiently, and is auditing the rest of the systems, it said.

It also announced that it would make changes to the page that shows whether the service is actually online. Even as Amazon Web Services and the websites that rely on it were falling over, its update page said that everything was fine – because that same page relied on S3 and so was affected by its own outage. It will now rely on different data centres so that it wouldn't buckle if just one went down, it said.

Independent

Other News

Hanoi approves controlled pilot of two technology projects

Hanoi approves controlled pilot of two technology projects

The two projects involve an intelligent autonomous transportation system developed by Phenikaa-X JSC and a technology for producing mixed rare-earth oxides with a purity of over 95% on a semi-industrial scale developed by the Mining Technology and Construction Consultancy JSC.

Banking industry in AI era: When data becomes competitive advantage

Banking industry in AI era: When data becomes competitive advantage

In just a few years, AI has evolved from an experimental technology into an “assistant” for many banks, supporting everything from customer service, credit approval, and fraud detection to risk management. However, as the technological gap between institutions narrows, a new question is emerging: What will determine banks' competitive advantage in the AI era?

Bac Ninh considers establishing AI centre of excellence

Bac Ninh considers establishing AI centre of excellence

A meeting has been held in Bac Ninh to discuss potential cooperation with Plug and Play in developing the northern province’s innovation ecosystem and establishing an artificial intelligence (AI) centre of excellence, with Standing Vice Chairman of the provincial People’s Committee Mai Son receiving a delegation from the global innovation platform.

Hanoi allows 24-month trial of autonomous vehicles at Hoa Lac

Hanoi allows 24-month trial of autonomous vehicles at Hoa Lac

The project aims to assess the safety, reliability and readiness of autonomous vehicles under real traffic conditions, as well as the effectiveness of autonomous public transport and compatibility with existing systems. Its results will contribute to developing technical standards, operating procedures and management models suited to Hanoi’s traffic conditions.

Singapore's experience suggests solutions to Vietnam's human resources challenge in AI era

Singapore's experience suggests solutions to Vietnam's human resources challenge in AI era

As Vietnam accelerates digital transformation and high-tech development, Singapore’s experience and the engagement of global technology companies demonstrate that human resources preparation must stay ahead of the curve. The ability to filter knowledge, collaborate, innovate and effectively use AI will increasingly become essential to helping workers adapt to a changing labour market and strengthening the competitiveness of Vietnam’s economy in the new era.

Protecting national security amid digital transformation, cyberspace

Protecting national security amid digital transformation, cyberspace

Cybersecurity, information security and data security risks must be identified and addressed early and from afar, with appropriate defensive measures ready to neutralise threats and protect national interests in cyberspace, said Lieutenant General Le Xuan Minh, Director of the Ministry of Public Security’s Department of Cybersecurity and Hi-tech Crime Prevention (A05).

Mekong Delta launches first robotics, AI research institute

Mekong Delta launches first robotics, AI research institute

Dr. Nguyen Van Quang, Rector of Nam Can Tho University, said the university is developing a value chain covering training, research, technology development, testing, technology transfer and commercialisation. The model is designed to link research more closely with practical needs, ensuring that technologies generate products and products create value for society.

Vietnam positioned to harness AI wave, World Bank report says

Vietnam positioned to harness AI wave, World Bank report says

The WB report identifies Vietnam as one of the few economies outside the European Union (EU) to have adopted a comprehensive AI regulatory framework as early as 2025. Vietnam applies a risk-tiered approach similar to that of the EU AI Act, imposing different obligations depending on whether an AI system poses “unacceptable,” “high,” “low” or “minimal” risks. The approach is viewed as a strategic step towards ensuring safe and transparent AI governance.

AI offers both opportunities, challenges for Vietnam's growth

AI offers both opportunities, challenges for Vietnam's growth

Dr. Ho Duc Thang, a full-time member of the National Assembly’s Committee for Cultural and Educational, said productivity is the key to sustaining double-digit economic growth, and AI has greater potential to improve productivity than any previous technology.

Top leader unveils people-centred vision for new development model

Top leader unveils people-centred vision for new development model

Vietnam's growth model, long fueled by low-cost labour, resource extraction, contract manufacturing, expanded investment, and capital accumulation, has run out of road. The country now needs a sweeping reform agenda to shift from extensive growth to a model driven by productivity, knowledge, science and technology, innovation, and digital transformation.