What is Pulse Wave? Hackers devise new DDoS attack technique aimed at boosting scale of assaults

The new attack method allows hackers to shut down targets’ networks for longer periods while simultaneously conducting attacks on multiple targets.

Hackers have begun launching a new kind of DDoS attack designed to boost the scale of attacks by targeting soft spots in traditional DDoS mitigation tactics. Dubbed “Pulse Wave”, the new attack technique allows hackers to shut down targeted organisations’ networks for prolonged periods while simultaneously conducting attacks on multiple targets.

The new attacks may render traditional DDoS mitigation tactics useless, experts say. Some of the pulse wave DDoS attacks detected lasted for days and “scaled as high as 350 Gbps”, according to security researchers at Imperva, who first spotted the new threat.

“Comprised of a series of short-lived pulses occurring in clockwork-like succession, pulse wave assaults accounted for some of the most ferocious DDoS attacks we mitigated in the second quarter of 2017,” Imperva researchers said in a report.

The researchers said they believe that the pulse wave technique was “purposefully designed” by “skilled bad actors” to boost hackers’ attack scale and output by taking advantage of “soft spots in hybrid ‘appliance first, cloud second’ mitigation solutions.”

Traditional DDoS attacks involve a continuous barrage of assaults against a targeted network, while pulse wave involves short bursts of attacks that have a “highly repetitive pattern, consisting of one or more pulses every 10 minutes”. The new attacks last for at least an hour and can extend to even days. A single pulse is large and powerful enough to completely congest a network.

“The most distinguishable aspect of pulse wave assaults is the absence of a ramp-up period — all attack resources are committed at once, resulting in an event that, within the first few seconds, reaches a peak capacity that is maintained over its duration,” the Imperva researchers said.

ulse wave takes advantage of appliance-first hybrid mitigation solutions by preying on the “Achilles’ heel of appliance-first mitigation solutions”, – the devices’ incapability of dealing with sudden powerful attack traffic surges.

The Imperva researchers said the emergence of pulse wave DDoS attacks indicates a significant shift in the attack landscape. “While pulse wave attacks constitute a new attack method and have a distinct purpose, they haven’t emerged in a vacuum. Instead, they’re a product of the times and should be viewed in the context of a broader shift toward shorter-duration DDoS attacks,” researchers said.

The Imperva researchers predicted that such attacks will continue, becoming more persistent and growing, boosted via botnets.

Source: http://www.ibtimes.co.uk/what-pulse-wave-hackers-devise-new-ddos-attack-technique-aimed-boosting-scale-assaults-1635423

 

  • 0

Why DDoS attacks show no signs of slowing down

Distributed Denial of Service (DDoS) attacks caused substantial damage to organisations across APAC and the world in the past year.

According to Neustar’s recent ‘Worldwide DDoS Attacks and Cyber Insights Research Report’, 84 percent organisations surveyed globally were hit by a DDoS attack in the last 12 months, with 86 percent of those organisations were hit multiple times.

The code used to cause these large outages was published openly, and soon after all sorts of attacks and variants of the original code were causing havoc around the world.

Detection is too slow

DDoS attacks are not only occurring more frequently but are also getting more difficult to detect.

Within APAC, more than half of organisations on average are taking at least three hours to detect an attack and nearly as many took another three hours to respond once an attack was detected.

Alarmingly, slow detection and response can lead to huge damages financially. Around half of all organisations stand to lose an average of $100,000 per hour of peak downtime during an attack. To exacerbate this, 40 percent of organisations hit were notified by their customers of the attacks.

Investment is increasing

The worrying figures above help explain why 90 percent of organisations are increasing their investments in DDoS defences, compared to the previous 12 months – up from 76 percent last year- despite the fact that 99 percent already have some form of protection in place.

The threats faced today, and those anticipated in the future, are clearly forcing organisations to completely reconsider the ways they are currently protecting themselves.

Mitigating against DDOS attacks

Effectively mitigating DDoS attacks has become crucial for organisations that want to avoid damaging financial and reputational loss. In order to combat attacks, organisations need to adequately understand the threat, quantify the risk and then create a mitigation plan that corresponds to their needs.

Whether it’s a large or small scale DDoS attack, to keep up with the growing threat, companies will need newer, adaptable, and scalable defences that include new technology and methodologies.

Developing a mitigation plan

Paying the cost for a DDoS mitigation that exceeds their requirements is like over insuring your car – you are paying a premium for a service that does not match your level of risk/potential loss.

Similarly, implementing a DDoS mitigation that does not cover the risk will likely lead to additional costs, resulting from greater organisational impact and additional emergency response activities.

Once the severity of the risk is understood, there are three key critical elements of producing a good mitigation plan that must be enacted: detection, response and rehearsal.

Detecting an attack

Fortunately, there are several technologies out there that can be used to monitor both the physical and cloud-based environment. An example is how organisations can use Netflow monitoring on border routers to detect a volumetric attack, or provide this data to a third-party for analysis and detection.

They can also look at using appliances to conduct automatic detection and response, again managed internally or by a third-party. In a cloud environment, organisations can choose between a vast array of cloud monitoring tools that allow them to identify degradation and performance, CPU utilisation and latency, giving an indication as accurate as possible of when an attack occurs.

Responding to an attack

The response plan to the attack must be scaled to the organisation’s risk exposure and technology infrastructure. For instance, an organisation operating in the cloud with a moderate risk exposure might decide on a cloud based solution, pay-on-occurrence model.

On the other hand, a financial services company that operates its own infrastructure will be exposed to more substantial financial and reputational risk. Such a company would ideally look for a hybrid solution that would provide the best time to mitigate, low latency and near immediate failover to cloud mitigation for large volumetric attacks.

Rehearsal of your mitigation plan

Regardless of the protection method being deployed, it’s good practice to rehearse it periodically. Periodic testing can not only eliminate gaps or issues in responding to a DDoS attack, but can also prepare the responsible owners to perform their required actions when an actual event occurs.

In summary, DDoS attacks aren’t showing any signs of slowing down anytime soon. The threats associated with DDoS attacks cannot be understated or underestimated. Moreover, by quantifying the risk to the organisation and implementing a right-sized mitigation solution, organisations can effectively and efficiently mitigate the risk of DDoS attacks.

Source: https://securitybrief.com.au/story/why-ddos-attacks-show-no-signs-slowing-down/

  • 0

BLIZZARD ENTERTAINMENT HIT WITH WEEKEND DDOS ATTACK

Blizzard Entertainment reported a crippling DDoS attack over the weekend creating chronic latency and connection issues for players of games Overwatch, World of Warcraft and others.

The DDoS attack has since subsided, according to Blizzard, but users are still grousing on Twitter over lingering connection issues and feature unavailability within some games.

Screen Shot 2017-08-15 at 09.45.51

The attacks began early Sunday with Blizzard acknowledging the issue on Twitter.

“We’re currently investigating an issue affecting our authentication servers, which may result in failed or slow login attempts,” Blizzard tweeted.

According to third-party service Down Detector, Blizzard experienced a sharp increase in network problems mid-day Sunday with users reporting an inability to log into games, server connection problems and some reporting the Blizzard Entertainment webpage appearing to be down.

No person or group has taken responsibility for the DDoS attack. Blizzard did not return a request to comment for this story.

“Competitive online games are an attractive target for DDoS offenders looking to create large-scale mayhem in hopes of gaining some internet notoriety,” said Igal Zeifman, a senior manager at security firm Imperva.

Zeifman told Threatpost that real-time gaming networks are attractive high-profile targets for hackers. “In the case of a real-time online game, even a small amount of latency—as a result of a technically ‘failed’ attack—is enough to cause major disruption to gamers looking for a completely responsive and immersive experience,” he said.

Zeifman suspects the hackers in this attack could have similar motives to Lizard Squad when it levied a 2014 DDoS attack against the PlayStation Network and Xbox Live. Soon after gaining notoriety for that attack, Lizard Squad advertised a DDoS attack tool that cost $6 a month.

“While notoriety is typically the motivating factor behind the attacks, sometimes it’s just a user with a beef against the game or a hacker simply trying to impress someone or group,” Zeifman said.

More recently, in June Final Fantasy 14’s servers experienced a wave of DDoS attacks that lingered into July, according to Square Enix. In that case, the hacker or group was also not identified.

The Blizzard attack coincided with problems with Blizzard customers using PayPal as a payment option.

It’s unclear if the weekend Blizzard service disruptions were related to earlier issue reported last Tuesday and Friday. For example, several news outlets reported long-than-normal queue times for Blizzard games on Tuesday. On Friday, Down Detector reported server connection issues, login problems and that the Blizzard website was down.

Source: https://threatpost.com/blizzard-entertainment-hit-with-weekend-ddos-attack/127440/

  • 0

World of Warcraft, Overwatch, Hearthstone and other games hit by DDoS

Games company Blizzard has reported on Twitter that: “We are currently monitoring a DDOS attack against network providers which is affecting latency/connections to our games.”  World of Warcraft, Overwatch, Hearthstone and other game servers are believed to have been hit.

At about 5pm last night Blizzard noticed Down Detector – which monitors online outages  -logging a huge upsurge of problems and 2800+ reports for Overwatch, World of Warcraft and several other Blizzard gaming services.

Commenting on the way that even failure to bring a service down completely has a severe impact on online games,  Igal Zeifman, director at Imperva Incapsula said in an email to SC: “Competitive online games are an attractive target for any DDoS offender looking to create large-scale mayhem in hope of some Internet notoriety. Moreover, such gaming networks are also particularly vulnerable to denial of service assaults because, unlike many other targets, they don’t need to be taken offline to become unusable.

“In the case of a real-time online game, even a small amount of  latency–as a result of a technically “failed” attacks–is enough to cause major disruption to gamers looking for a completely responsive and immersive experience. This is exactly what is happening in this case. Even if some users are able to log in, the latency they experience still makes Overwatch unplayable.”

Source: https://www.scmagazineuk.com/world-of-warcraft-overwatch-hearthstone-and-other-games-hit-by-ddos/article/681508/

  • 0

Kids these days: the 16-year-old behind 1.7 million DDoS attacks

Teenagers have typically not been known as the most motivated demographic, napping through classes and slouching through shifts at McDonald’s.

While yelling at a 16-year-old four times just to get him to unload the dishwasher is annoying, consider the other end of the spectrum: the ambitious 16-year-old who earned over $500,000 USD by building a DDoS stresser responsible for 1.7 million attacks, causing millions of dollars in damages.

It’s cool Brayden, you can unload the dishwasher later.

Dirty dealings

A successful distributed denial of service or DDoS attack is one in which a website or online service is overwhelmed by malicious traffic or requests, pushing the site or service offline so it’s unavailable to its users. DDoS attacks have been big news the last few years. Big news to website owners who have had users frustrated by downtime, to business owners who have suffered reputation damage and monetary losses, to the public at large who have been unable to use websites and services big and small because of these attacks, and big news to the media itself who have been devoting headlines to the ever-growing scourge of attacks.

One of the main reasons for the increase in attacks has been DDoS for hire servers, otherwise known as booters or stressers. For as little as a few dollars, anyone with an internet connection can buy access to a service that allows them to aim a DDoS attack at the targets of their choosing. Stressers are so named because they masquerade as a legitimate tool, one that stresses a server to test its reliability.

This is where Adam Mudd comes in.

In the Mudd

When Adam Mudd was just 16 years old he went to work on the computer in his bedroom and created what he called the Titanium Stresser. Mudd himself carried out 594 distributed denial of service attacks, including an attack against his former college, but those nearly 600 attacks were but a drop in the bucket compared to how busy his stresser got when he opened it up as a DDoS for hire service.

In just over two years the Titanium Stresser racked up 112,000 registered users who launched 1.7 million DDoS attacks against 660,000 IP addresses. There were obviously many repeat targets amongst those 660,000 IP addresses, perhaps most notably the company behind the online game RuneScape which was hit 25,000 times and led to the company spending roughly $10 million in mitigation efforts. Other notable targets of the Titanium Stresser included Sony, Xbox Live, Microsoft and Team Speak. Mudd reportedly earned over $500,000 from his stresser service.

It all came to an end for Mudd in March of 2015 when the police arrived at his parents’ house. Mudd refused to unlock his computer until his father intervened. He has since pleaded guilty to three charges under the United Kingdom Computer Misuse Act, and one charge of money laundering. He was sentenced to 24 months in jail.

The big picture

Mudd was nothing more than a teenager in the bedroom of his parents’ house, yet his stresser service caused millions of dollars in quantitative damages and untold further damages when it comes to lost productivity, lost user loyalty and lost revenue in both the short and long term. There are Adam Mudds all over the world, many more experienced, running stresser services that are just as successful as the Titanium Stresser and even more so.

Further, while Mudd’s arrest and conviction is a success for law enforcement, he joins a list of recent DDoS-related arrests that include members of the famed Lizard Squad, owners of the vDos botnet, and three dozen patrons of stresser services. Hackforums, the biggest hacking forums in the world, also recently banned DDoS for hire services. All seemingly good things. Yet the number of DDoS attacks being perpetrated hasn’t gone down. When the FBI or Interpol shuts down a stresser service, another stresser service simply scoops up its customers.

The lesson here has to be that DDoS attacks can be perpetrated by anyone and aren’t going anywhere anytime soon. With stresser services so affordable and accessible, almost every website on the internet is a potential target, and potentially a repeat target. Without professional DDoS protection, websites will be left picking up the pieces and paying exorbitant sums in order to do so.

Source: http://www.bmmagazine.co.uk/in-business/kids-days-16-year-old-behind-1-7-million-ddos-attacks/

  • 0

Libertarian Site Suffers DDoS Attack After Supporting Google Worker

Quillette Magazine, a small but respected libertarian publication based in Australia, suffered a DDoS attack Tuesday after publishing an article supportive of James Damore, the fired Google memo writer.

The attack, which crashed the site for a day, came after Quillette published the opinion of four scientists on the Google memo. The scientists found that the conservative Google employee’s views on gender differences were supported by substantial scientific evidence.

The Google memo’s “key claims about sex differences are especially well-supported by large volumes of research across species, culture,” wrote Geoffrey Miller, a professor of evolutionary psychology at the University of New Mexico, explaining that the memo “is consistent with the scientific state of the art on sex differences.”

“Among commentators who claim the memo’s empirical facts are wrong, I haven’t read a single one who understand sexual selection theory, animal behavior, and sex differences research,” Miller added.

Deborah Soh, who has a PhD in sexual neuroscience and works as a Toronto-based science writer, concurred with Miller. “Sex differences between women and men—when it comes to brain structure and function and associated differences in personality and occupational preferences—are understood to be true, because the evidence for them (thousands of studies) is strong.”

“This is not information that’s considered controversial or up for debate; if you tried to argue otherwise, or for purely social influences, you’d be laughed at,” Soh said.

Unfortunately, liberal-hacker-activists couldn’t handle the truth, and Quillette’s website took an arrow to the knee. Claire Lehmann, the founder of Quillette, told PJ Media that her website was especially susceptible to attack.

While there are many programs that can be used to protect against DDoS attacks (which are when hackers flood websites with traffic to crash it), Claire said she didn’t have any.

“I’m a small site and my technical skills are not at a high level, so I was unaware that I should have had these protections. Apparently they are fairly standard,” she told PJ Media.

Her site, which has received endorsements from well-known figures such as Charles Murray and Richard Dawkins, has a history of publishing science-based journalism, but this is the first time they’ve suffered a DDoS attack, Lehman says. (Disclosure: I’ve written a few articles on higher education for them. Small world.)

Lehmann, whose site has been dedicated to supporting alternative viewpoints since it launched in 2016, said her work is crucial to helping people see the truth behind things. “It’s important to hear alternative viewpoints so that we can work out what is the truth, and not merely consensus,” Lehmann said.

“Over the past few years, both academic and media institutions have become highly conformist. And we know that groupthink leads to blindspots, which makes us unable to see what is actually true.”
Source:https://pjmedia.com/trending/2017/08/09/libertarian-site-suffers-ddos-attack-after-supporting-google-worker/
  • 0

Long-lasting DDoS attacks are back

Kaspersky report reveals the return of major DDoS threats, which are now also lasting longer than ever before.

Long-lasting DDoS attacks are back, and they’re harder than ever, new research has claimed.

According to a report from Kaspersky Lab, the second three months of 2017 saw a DDoS attack last more than 11 days – 277 hours straight.

That’s a 131 per cent increase compared to Q1 2017, and a record for the year so far.

The report also says that duration was not the only key feature of DDoS attacksthis quarter, identifying a ‘dramatic change’ in the geography of these threats. The top 10 most affected countries are China, South Korea, USA, Hong Kong, UK, Russia, Italy, the Netherlands, Canada and France — with Italy and the Netherlands replacing Vietnam and Denmark among the top targets in Q1.

Al Jazeera, Le Monde and Figaro were the biggest targets, alongside Skype servers. Criminals also tried to manipulate cryptocurrency prices through DDoS. Bitfinex was attacked simultaneously with the launch of trading in a new cryptocurrency called IOTA token.

“Nowadays, it’s not just experienced teams of hi-tech cybercriminals that can be Ransom DDoS-attackers. Any fraudster who doesn’t even have the technical knowledge or skill to organise a full-scale DDoS attack can purchase a demonstrative attack for the purpose of extortion. These people are mostly picking unsavvy companies that don’t protect their resources from DDoS in any way and therefore, can be easily convinced to pay ransom with a simple demonstration,” comments Kirill Ilganaev, head of Kaspersky DDoS protection at Kaspersky Lab.

Source: http://www.itproportal.com/news/long-lasting-ddos-attacks-are-back/

  • 0

FCC says its cybersecurity measures to prevent DDoS attacks must remain secret

The FCC has provided a few — very few — details of the steps it has taken to prevent attacks like the one that briefly took down its comment system in May. The agency has faced criticism over its secrecy regarding the event, and shows no sign of opening up; citing “the ongoing nature of the threats,” to reveal its countermeasures would “undermine our system’s security.”

These cryptic comments are the first items of substance in a letter (PDF) sent to the House Energy and Commerce and Government Reform committees. Members thereof had sent letters to the FCC in late June asking what solutions it was implementing to mitigate or prevent future attacks.

A cover letter from FCC Chairman Ajit Pai emphasizes the fact that millions of comments have been filed since, including 2 million in the 4 days following the attack. He writes that the Commission’s IT staff “has taken additional steps… to ensure the ongoing integrity and resiliency of the system.”

What those steps are, however, he did not feel at liberty to say, except that they involve “commercial cloud providers” and “internet-based solutions.” Since the comment filing system is commercially cloud-hosted, and the system is fundamentally internet-based, neither of these descriptions is particularly revelatory.

It’s not the security, it’s the communication

The issue, however, isn’t that we are deeply afraid that another hacker will take down the system. After all, basic rate limiting and some analytics seem to have done the job and allowed record numbers of comments immediately after the attack stopped. The FCC was still writing reports and calling experts at the time the system had returned to full operation.

The issue is the FCC’s confusing and misleading handling of the entire thing.

The nature and extent of the attack is unclear — it’s described in a previous letter to concerned senators as a “non-traditional DDoS attack.” Supposedly the API was being hammered by cloud-based providers. What providers? Don’t they have records? Who was requesting the keys necessary to do this?

Very little has been disclosed, and even requests of information circumstantialto the attacks have been denied. What is so sensitive about an analysis of the network activity from that period? Petitioners seeking to see communications pertaining to the attack were told much of the analysis was not written down. Even the most naive internet user would find it hard to believe that in a major agency of a modern bureaucracy, a serious attack on its internet infrastructure, concerning a major internet policy, would fail to be discussed online.

 The FCC also says it consulted with the FBI and agreed that the attack was not a “significant cyber incident” as such things are defined currently in government. For the curious:

A cyber incident that is (or group of related cyber incidents that together are) likely to result in demonstrable harm to the national security interests, foreign relations, or economy of the United States or to the public confidence, civil liberties, or public health and safety of the American people.

Okay, that seems reasonable. So why is it being kept under wraps? Why are the countermeasures, which are probably industry standard, unable to be disclosed? How would disclosing the details of those security countermeasures undermine those systems?

If it’s the “ongoing threat,” what is the threat exactly if not the pervasive threat of hacking faced by any public website, service or API? Have there been follow-up attacks we haven’t been informed of? The investigation is also ongoing, but in that case how could it fail to produce written records for FOIA requests like those already submitted?

The more the FCC drags its feet and stammers out non-answers to simple questions regarding what it itself has categorized a non-major attack that happened months ago and did not significantly affect its systems, the less we trust what it does say.

Concerned senators, representatives and others are not going to stop asking, however. Let’s hope whatever the FCC seems unwilling to share comes out before it ceases to be relevant. It would be a shame, for instance, to receive a full report on hackers bent on supporting one side of the net neutrality argument… the day after the FCC votes on the issue.

Source: https://techcrunch.com/2017/07/31/fcc-says-its-cybersecurity-measures-to-prevent-ddos-attacks-must-remain-secret/

  • 0

DDoS Extortionist Who Posed as Anonymous Hacker Arrested in the US

On Friday, US authorities arrested a man on charges of launching DDoS attacks and making death and bomb threats against several targets including Leagle.com, the Sydney Morning Herald, the Canadian Broadcasting Corporation (CBC), Metro News Canada, the official website of the Canadian government, and others.

The man’s name is Kamyar Jahanrakhshan, a man born in Iran, who later obtained US citizenship in 1991, and then a permanent residency in Canada in 1995.

Following two criminal cases of theft in 2005 and fraud in 2011, Jahanrakhshan was deported from Canada to the US in 2014.

Suspect wanted his criminal past erased from the Internet

According to court documents obtained by Bleeping Computer, after his deportation, Jahanrakhshan started sending emails to online websites that had written articles or had copies of his past criminal record.

The first organization that Jahanrakhshan targeted was Leagle.com, a website that offers copies of court opinions and decisions. In the beginning, Jahanrakhshan contacted the site’s team from his personal email address, asking them nicely to remove copies of past court decisions mentioning his name on the premise that it was tarnishing his reputation and violating his privacy.

When the Leagle team refused, the suspect even offered to pay a $100 fee to have the documents removed. When Leagle refused again, Jahanrakhshan — who also used the name “Andrew Rakhshan” — sent them a threatening email saying he made friends with dangerous hackers and they should heed his final warning.

Suspect poses as group of Anonymous hackers

After Leagle had ignored him again, US authorities say Jahanrakhshan launched a DDoS attack on the site’s servers and sent an email from a Yahoo account posing as a member of the Anonymous hacker collective.

Copy of the message the suspect sent Leagle
Copy of the message the suspect sent Leagle

US authorities say they found evidence linking Jahanrakhshan to this email account, but also to others emails linked to other DDoS extortions.

Because they couldn’t handle the DDoS attack, Leagle eventually removed a decision that Jahanrakhshan had asked. The DDoS attacks stopped after.

Initial success leads to more DDoS extortions

The FBI says that after having forced Leagle to remove a damaging report on his past criminal record, Jahanrakhshan moved on to other targets.

During 2015 and 2016, Jahanrakhshan would allegedly engage in a similar behavior and take aim at other online publications that had written articles on his past crimes, such as the Sydney Morning Herald, Canadian Broadcasting Corporation (CBC), Metro News Canada, and the official website of the Canadian government.

To put extra pressure on his targets to remove damaging articles, authorities say he also launched DDoS attacks on the websites of customers advertising on CBC and Canada.com — Postmedia and the Inspiration Foundation.

Seeing that all this failed and none of his targets removed the incriminating articles, Jahanrakhshan also moved on to sending bomb threats at the offices of targeted organizations and death threats on family members of employees working for the targeted organizations.

He was arrested this week and arraigned in court on Friday. The suspect, if found guilty, could face up to five years in prison and a fine of up to $250,000.

The damaging articles Jahanrakhshan was trying to take down described how he used fake credit cards to buy a fleet of luxury cars and a boat

Source: https://www.bleepingcomputer.com/news/security/ddos-extortionist-who-posed-as-anonymous-hacker-arrested-in-the-us/

  • 0

Don’t ban the bots

I do a lot of DDoS related research online, which results in a lot of DDoS protection related spam/offers. A trend I have seen gaining popularity lately is “ban the bots”.

These emails contain a lot of emotionally charged language trying to persuade the reader that bots are destroying the internet, wasting your bandwidth and pillaging your website (and how for a modest monthly fee they can keep the digital invaders at bay). I couldn’t disagree more. For the most part I like bots. Bots save me a ton of work and allow me to the focus on tasks that are meaningful to me. The only reason that search engines, hotel booking sites, and social media sites operate so successfully (or at all) is because of bots.

These advertisements do acknowledge there are some good bots out there, while stressing the need to block the bad bots. I thought I’d pull some numbers from traffic running through our system. I was pleasantly surprised, as a DDoS protection service I was expecting to see more malicious bots than legitimate but what I found was 85% of the bot traffic is classified as good: SES (which stands for Search Engine Spiders, but is a general list of the known good bots) which we don’t want to block, and XSE which contains alternate Spiders and bots that while legitimate can cause impact on some websites.

Screen Shot 2017-07-27 at 15.38.10

The other 15% of traffic is from hosting companies, ISPs, and commercial traffic from unknown bots. This traffic is not automatically bad, but hidden somewhere in there are the malicious bots and scrapers which we do want to block. This is where the philosophy “ban the bots” makes things more complicated than it needs to be, because while it is a trivial matter to find and locate bots, it focuses you on the actor not the action. Don’t ban the bots, ban the malicious actions. If you design your web security to defend against malicious actions it shouldn’t matter whether they are from bots or not. At DOSarrest this is what we do, we create

special features to focus on the malicious bot traffic and apply them to customer configurations and leave the good bots alone.

In fact, I’ll go one step further: don’t ban the bots, help the bots. Because while I disagree with the conclusion the facts are not wrong, bots do consume more than a trivial amount of resources. By helping the bots find the content they are looking for you can reduce the impact on your site and possible improve your overall ranking.

Your first goal is getting the bots to your content in as few requests as possible, and at the same time stopping the bots from crawling pages you don’t need (or want) to show up in search results. Most modern sites have dynamic, pop-up, hidden menus that require multiple javascript and CSS resources to properly render. They might look fantastic, but a bot isn’t interested in the aesthetics of your site, they are looking for content. A sitemap is a great tool for linking all the content you want to emphasize without a bot having to navigate through a bunch of complicated dynamic resources. Then there are the rest of the pages in your site, things that are useful to your users but not things that need to appear in the search rankings, login pages, feedback forms, etc. Use robots.txt file or ‘noindex’ meta tags to direct the bots not to bother with these pages.

Your sitemap and robots.txt will help bots find the resources you want them to find, and avoid the ones you don’t. This will help lighten the load on your webserver, but won’t necessarily help your site ranking. The number one thing they are looking for is quality content. But searchbots also look for good performing sites. Too many errors or slow responses will negatively impact your ranking in a big way. The answer here is caching. Many bots, googlebot included, do full page downloads when indexing your site. They are looking for javascript and CSS files, images and PDFs, or whatever resources you’ve linked. Most of these resources are static and can be served up out of a CDN. Not only will this alleviate the load on your server, but the performance improvement will make all your quality content that much more appealing to the bots.

Sean Power

Security Solutions Architect

DOSarrest Internet Security

Source: https://www.dosarrest.com/ddos-blog/don-t-ban-the-bots/

 

  • 0