Plugin Sale! Save 15% on pro plugins with discount code: FALL2020
Web Dev + WordPress + Security

All the little .txt files you can put in the root directory of your website

The ones I know of:

ads.txt
humans.txt
robots.txt
security.txt

This site makes use of robots.txt and humans.txt. I don’t need ads.txt because 3rd-party ads aren’t currently running on the site, and security.txt seems not necessary as the site’s contact form is easy enough for anyone to find.

I’m guessing there are other “little text files” out there, but I am super busy building a bookstore subdomain for the site with every spare minute. So for now I’ll post the files that I’m aware of; I’m sure in time more will be found and added to the list.

Also: how are people referring to these files? Surely there is something better than “all the little dot-text files you can put in the root directory of your website”.

I think..

Eventually there will be all sorts of these little text files. Anyone can start a text file thing. Like contact.txt for contact information. Or help.txt for support. It could even go beyond site meta and do like friends.txt or any-useful-bit-of-information.txt.

Jeff Starr
About the Author
Jeff Starr = Creative thinker. Passionate about free and open Web.
USP Pro: Unlimited front-end forms for user-submitted posts and more.

14 responses to “All the little .txt files you can put in the root directory of your website”

  1. I only allow a few text and XML files to be accessible on my server(s) anyway, and disallow any others:

    RewriteCond %{REQUEST_URI} \.txt
    RewriteCond %{REQUEST_URI} !^/(humans|robots|)\.txt$ [NC]
    RewriteRule .* - [R=404,L]
    
    RewriteCond %{REQUEST_URI} \.xml
    RewriteCond %{REQUEST_URI} !^/(sitemap|search)\.xml$ [NC]
    RewriteRule .* - [R=404,L]

    To me, anything else is treated as “probing”. Plus, this puts a stop to one of the methods of plugin-enumeration, like what is used by WPScan.

    – Jim S.

    • Jeff Starr
      Jeff Starr 2020/02/20 1:10 pm

      Elegant technique for sites with no other .txt or .xml files. Thanks for sharing.

      Also about the code:

      1) There is an extra pipe | in the 2nd condition, will cause an error.

      2) Also, did you include any <IfModule> logic? It looks like WordPress removed them (despite my programmatic efforts to always allow).

      • Huh?

        A pipe error?

        It may be possible that the “!^/” needs to be changed to: “!(^|/)”, because the “or” pipe should work. I have a bit more complex REGEX in my own site’s htaccess file. This was originally intended to also be usable on a “multi-site, URI-style” site.

        I’ll check it further anyway.

        – Thanks.

        – Jim S.

      • Jeff Starr
        Jeff Starr 2020/02/24 1:23 am

        Yeah you have this:

        RewriteCond %{REQUEST_URI} !^/(humans|robots|)\.txt$ [NC]

        Look at the pipe symbol after robots.

  2. Rick Beckman 2020/02/20 9:13 pm

    Have you heard of placing these files in /.well-known? I’ve seen that with increasing frequency, and apparently it has an RFC. Any idea on how worthwhile separating these files away like this is?

    Presumably the root directory URIs for such files could be 301 redirected to the “well known” directory easily enough using .htaccess.

    • Jeff Starr
      Jeff Starr 2020/02/21 8:37 am

      I’ve heard of .well-known and redirecting requests for common/related files, but have not heard of placing the files inside of the otherwise hidden .well-known directory. It’s not something I would ever do for any of my own sites, but there is some logic to be found by putting all of the miscellaneous text or whatever files in the same location. I think it might be a bad idea though to make such location a hidden directory, if any of the “well-known” files are meant for human consumption (e.g., ads.txt, humans.txt, security.txt).

    • If you use Let’s Encrypt Certificates in your site, and you use the Apache2 (or whatever SAPI) server for your website-owner validation, you may see in your access logs the Let’s Encrypt bot tries to access the validation hash files in /.well-known/acme. So I have had to craft a rather convoluted way to protect this folder AND allow access to only certain files within it.

      – Jim S.

  3. Jeff you should learn about the .txt file mta-sts and all the email security settings, every website should setup as standard.

    p.s. there are 4 major email security things to setup in the dns records of every website.

    I won’t name them, instead I leave you to research and find them for yourself.

    • Jeff Starr

      Very interesting, thanks Joe. I found this, looks like it requires a lot of time to read and implement everything. For the “4 major email security things” for DNS, are you talking about SPF, DKIM, and DMARC? What’s the fourth one?

      • Actually they are all very easy to setup! To confirm the four are: SPF, DKIM, DMARC and MTA-STS.

      • Jeff Starr

        Right, that makes sense. Will investigate further, thanks Joe.

      • On the subject of DKIM,

        Still refining an automated technique. Otherwise, very simple to implement. Far easier than for CSP-headers anyway.

        – Jim S.

  4. Maybe add security.txt? Many security researches use it to contact the website owners about a breach.

Comments are closed for this post. Something to add? Let me know.
Welcome
Perishable Press is operated by Jeff Starr, a professional web developer and book author with two decades of experience. Here you will find posts about web development, WordPress, security, and more »
USP Pro: Unlimited front-end forms for user-submitted posts and more.
Thoughts
Air finally clearing here in WA. Feeling grateful to breathe again. #oxygenmatters
Past week here in WA state has been hellish. So much smoke, like living in a chimney.
Now in September, I’m where I wanted to be in March.
Spent some time updating my article on unsafe characters, once again current with latest IETF specification.
Just realized that “Neo” is an anagram for “One”. As in, “he is the One” (The Matrix).
To get VLC app to load all songs (including subfolders), go to Preferences ▸ Show All ▸ Playlist ▸ Subdirectory behavior ▸ Expand.
Switching from PhotoShop to Affinity Photo is one of the most liberating work-related things I've done in 20 years.
Newsletter
Get news, updates, deals & tips via email.
Email kept private. Easy unsubscribe anytime.