Welcome to the new design! Please report any bugs or issues, thanks :)
Web Dev + WordPress + Security

All the little .txt files you can put in the root directory of your website

The ones I know of:

ads.txt
humans.txt
robots.txt
security.txt

This site makes use of robots.txt and humans.txt. I don’t need ads.txt because 3rd-party ads aren’t currently running on the site, and security.txt seems not necessary as the site’s contact form is easy enough for anyone to find.

I’m guessing there are other “little text files” out there, but I am super busy building a bookstore subdomain for the site with every spare minute. So for now I’ll post the files that I’m aware of; I’m sure in time more will be found and added to the list.

Also: how are people referring to these files? Surely there is something better than “all the little dot-text files you can put in the root directory of your website”.

I think..

Eventually there will be all sorts of these little text files. Anyone can start a text file thing. Like contact.txt for contact information. Or help.txt for support. It could even go beyond site meta and do like friends.txt or any-useful-bit-of-information.txt.

Jeff Starr
About the Author
Jeff Starr = Creative thinker. Passionate about free and open Web.
The Tao of WordPress: Master the art of WordPress.

12 responses to “All the little .txt files you can put in the root directory of your website”

  1. I only allow a few text and XML files to be accessible on my server(s) anyway, and disallow any others:

    RewriteCond %{REQUEST_URI} \.txt
    RewriteCond %{REQUEST_URI} !^/(humans|robots|)\.txt$ [NC]
    RewriteRule .* - [R=404,L]
    
    RewriteCond %{REQUEST_URI} \.xml
    RewriteCond %{REQUEST_URI} !^/(sitemap|search)\.xml$ [NC]
    RewriteRule .* - [R=404,L]

    To me, anything else is treated as “probing”. Plus, this puts a stop to one of the methods of plugin-enumeration, like what is used by WPScan.

    – Jim S.

    • Jeff Starr
      Jeff Starr 2020/02/20 1:10 pm Reply

      Elegant technique for sites with no other .txt or .xml files. Thanks for sharing.

      Also about the code:

      1) There is an extra pipe | in the 2nd condition, will cause an error.

      2) Also, did you include any <IfModule> logic? It looks like WordPress removed them (despite my programmatic efforts to always allow).

      • Huh?

        A pipe error?

        It may be possible that the “!^/” needs to be changed to: “!(^|/)”, because the “or” pipe should work. I have a bit more complex REGEX in my own site’s htaccess file. This was originally intended to also be usable on a “multi-site, URI-style” site.

        I’ll check it further anyway.

        – Thanks.

        – Jim S.

      • Jeff Starr
        Jeff Starr 2020/02/24 1:23 am

        Yeah you have this:

        RewriteCond %{REQUEST_URI} !^/(humans|robots|)\.txt$ [NC]

        Look at the pipe symbol after robots.

  2. Have you heard of placing these files in /.well-known? I’ve seen that with increasing frequency, and apparently it has an RFC. Any idea on how worthwhile separating these files away like this is?

    Presumably the root directory URIs for such files could be 301 redirected to the “well known” directory easily enough using .htaccess.

    • Jeff Starr
      Jeff Starr 2020/02/21 8:37 am Reply

      I’ve heard of .well-known and redirecting requests for common/related files, but have not heard of placing the files inside of the otherwise hidden .well-known directory. It’s not something I would ever do for any of my own sites, but there is some logic to be found by putting all of the miscellaneous text or whatever files in the same location. I think it might be a bad idea though to make such location a hidden directory, if any of the “well-known” files are meant for human consumption (e.g., ads.txt, humans.txt, security.txt).

    • If you use Let’s Encrypt Certificates in your site, and you use the Apache2 (or whatever SAPI) server for your website-owner validation, you may see in your access logs the Let’s Encrypt bot tries to access the validation hash files in /.well-known/acme. So I have had to craft a rather convoluted way to protect this folder AND allow access to only certain files within it.

      – Jim S.

  3. Jeff you should learn about the .txt file mta-sts and all the email security settings, every website should setup as standard.

    p.s. there are 4 major email security things to setup in the dns records of every website.

    I won’t name them, instead I leave you to research and find them for yourself.

    • Jeff Starr
      Jeff Starr 2020/02/21 11:43 am Reply

      Very interesting, thanks Joe. I found this, looks like it requires a lot of time to read and implement everything. For the “4 major email security things” for DNS, are you talking about SPF, DKIM, and DMARC? What’s the fourth one?

      • Actually they are all very easy to setup! To confirm the four are: SPF, DKIM, DMARC and MTA-STS.

      • Jeff Starr

        Right, that makes sense. Will investigate further, thanks Joe.

      • On the subject of DKIM,

        Still refining an automated technique. Otherwise, very simple to implement. Far easier than for CSP-headers anyway.

        – Jim S.

Leave a reply

Name and email required. Email kept private. Basic markup allowed. Please wrap any small/single-line code snippets with <code> tags. Wrap any long/multi-line snippets with <pre><code> tags. For more info, check out the Comment Policy and Privacy Policy.

Subscribe to comments on this post

Welcome
Perishable Press is operated by Jeff Starr, a professional web developer and book author with two decades of experience. Here you will find posts about web development, WordPress, security, and more »
Banhammer: Protect your WordPress site against threats.
Thoughts
Checking in to anyone listening. Stay safe. Pay attention. Don't get lazy.
What's up with Plesk UI lately? Especially on Chrome it looks just awful, all kinds of broken. Come on Plesk devs get it together.
Things get stressful, I try to pray. Not always easy, but always helps to relax and regain focus.
Nice new speed checker at fastorslow.com.
Easy way to exclude certain tests from WP Site Health: Site Health Tool Manager
Excellent (and free) tool for getting tons of site SSL infos: whynopadlock.com
Everyone just stay home and hide forever. Brilliant idea.