Website Attack Recovery
Recently, every website on our primary server was simultaneously attacked. The offending party indiscriminately replaced the contents of every index file, regardless of its extension or location, with a few vulgar lines of code, which indicated intention, identity, and influence.
Apparently, the attack occurred via Germany, through a server at the University of Hamburg (uni-hamburg.de). This relatively minor attack resulted in several hours of valuable online education. In this article, it is our intention to share experience with website attack recovery. This article is aimed at website developers, designers, and administrators.
After an attack
After an attack, begin with an overall damage assessment, surveying each of your sites. The extent of the damage will vary and may require external resources. If you have many sites, take notes. As you investigate the aftermath, capture screenshots and save offline copies of corrupted files. The idea is to collect evidence and information.
First and foremost, remain calm and stay smart. Email your server administrator and share any critical information (“Hey! I’ve been h4x0rd!!”). Notify any clients, customers, or subscribers immediately by sharing information in reassuring fashion. Additionally, if the damage is extensive, upload a default.html file with a soothing, explanatory public message. Then use an htaccess trick to redirect visitors.
Survey the damage
A comprehensive survey of the damage is critical. Examine code, files, settings, parameters, databases, scripts — everything. As you begin to understand the nature of the attack, do some research. Assume the worst — a seemingly minor attack could turn out to be quite extensive. Use a reverse IP tool to check for damage on neighboring domains on your server. Determine the specificity of the attack. If the attack is directed at your sites exclusively, proceed with extreme caution — someone could be watching your every move. In general, the more specific the attack, the greater the immediate threat.
Of course, with high-profile sites, time is of the essence. Take as much time as possible to research the attack before taking any restorative actions. Search the internet for any key phrases present after the attack. If no text is associated with the attack, describe the aftermath and search for related key terms and phrases.
Investigation
Once you begin to understand the attack, it is time to dig deep into logs. Hopefully, you have access to your server, cPanel, or some other online server interface. Investigate your error logs, access logs, and statistics logs. Look for unusual behavior, unexpected errors, and patterns of access indicative of an attack. Extensive stays (look for “Visits Duration” data) at URL’s containing forms or other input fields are prime suspects. Also look for activity or access of unlisted pages, scripts, or other content. Remember, take notes, screenshots, and so forth.
Determining time of attack will enable you to pinpoint a range of IP addresses in the statistics or access logs. For example, say you know that the attack happened on Wednesday, July 19th at approximately 1:30PM. From your stats log, order the Hosts/IP Address list by “date/time” and copy that particular range of IP addresses to your note file for later search. You want to scour as much IP data as possible.
Research
Once your notes are overflowing with juicy leads, start searching. With any luck, you will find something useful. Perhaps others have shared with similar experiences. Maybe the attack is defined in the Wikipedia. Better yet, the attacker(s) may have a website of their own!;) Any URL’s or IP’s associated with the attack should be investigated via a lookup/reverse lookup service. Comb thru the lookup results for keywords, names, and addresses and then search for them. Continue searching, knocking, and opening.
As you search, many “doors” will be opened to you. If you are using a browser equipped with tabbed browsing (e.g., Firefox), open every lead in a new tab and continue digging. It is advisable to have a plugin like Tab Save & Restore to enable you to restore your tabs should they crash. After you feel confident with your tab set, minimize that browser window and open a new one. If you surf tabless or need to scoot, create a new folder and save a nice set of good old-fashioned bookmarks.
By this time, you should have a solid grasp on the nature of the attack, the extent of damage, and hopefully an idea of the attacker’s identity (or at least their IP or URL). Using your new browser window, open new tabs and prepare to restore your websites, prioritized accordingly. You may also wish to download a copy of the compromised site for further research. Running an online virus scan beforehand is recommended.
Database restoration
Static sites not requiring database interaction should be the easiest to restore, assuming you are wise and periodically produce secure backup copies of your online work. Restore static sites by first deleting everything in the site’s directory. However, before uploading your restore, double-check the integrity of your site’s server parameters to ensure proper functionality and security. For example, check htaccess, indexes, and other settings. Then, after all is well, upload your website and check for functionality.
Dynamic sites may prove more difficult, depending on the nature of the attack, extent of damage, and structure/functionality of the site itself. If your site relies on a database, check it first. If anything looks odd in the database, create a backup of the “suspect” and then restore it with a previous, “non-suspect” copy. Or, if that is unreasonable or impossible, sleuth the database and manually restore any corrupted regions. Remember people, backups!
File restoration
Once you are confident with the database restoration, it is time to work with the dynamic site files. Before deleting everything, try restoring only the damaged files. The goal here is to establish a connection with the database and thus site functionality in general. If this works, it may not be necessary to restore the unaffected files, unless you feel it necessary “just to be sure.” If selective file replacement fails to remedy the problem, break out with the “brute force” method: delete everything and start over. With God’s blessing, it is only a matter of time until everything is once again online, operational, and open for business.
As you restore your websites, create fresh backup copies of everything: files, databases, scripts, etc. Generally, video files, audio files, and other large files do not need to be copied, especially if bandwidth is factor. Nonetheless, with a fresh assembly of restored websites, a complete archive of backup copies, and a fistful of recovery notes, you are on your way to completion and continued forward progress.
Recovery follow-up procedures include following through with your myriad searches, recording information, and taking any necessary post-operative actions to ensure site security.
Post-op
Returning to your open browser window, continue investigations until everything becomes perfectly clear. Return to the basic questions and try answering as many as possible: who attacked? what was attacked? where was the attack carried out? when was the attack carried out? why was the attack carried out? how was the attack accomplished? It is unlikely that you will achieve complete enlightenment, but the more answers the better.
As you gather information, remember to record all notes and keep them offline and private. The end result of this game should be a learning experience for the sake of improving understanding and better securing your sites against possible future attacks.
Prevention
Whereas improving the security inherent in specific scenarios is beyond the scope of this article, there are several important security measures that are generally applicable. First, establish a schedule for password maintenance. Create an excel sheet and get serious about password rotation. Create strong passwords, using memorable alphanumeric phrases for frequently used passwords and random alphanumeric strings for database users and other automated passwords.
Beyond passwords, look at file permissions. Although server attacks may override such settings, setting file permissions to “read-only” wherever possible will definitely work in your favor. Also, ensure that there are no vulnerable access points within the site itself. For example, inline contact forms are often exploited via various MySQL injections or PHP exploits (e.g., passthru() and system()). If you have the IP address or range of the perpetrators, there are several ways of denying them access (at least temporarily) to your site. Apache’s httpd.conf and htaccess work well for this.
Follow up
Lastly, when reporting news or info of the attack on your blog or elsewhere on the internet, omit any text associated with the attack itself. Attackers often enjoy reveling in search results that reveal public discussion and/or outcry of their attacks. Especially avoid using any unconventional or slang terms such as aliases or other strings of esoteric text left in the attack. If your online resource simply must include “searchable” text quoted from attack residuals, employ a crafty screenshot or use some .png/.gif/.fla-flavored text instead.
And finally, avoiding the use of “searchable” text associated with the attack will also prevent “seek-and-destroy” retribution for flaming rants or other incendiary dialogue. People who enjoy reading about themselves or their work are best served with a cold dish of apathy and indifference. No response, in my opinion, is the best response.
