• mozz@mbin.grits.dev
    link
    fedilink
    arrow-up
    14
    ·
    3 months ago

    I know this because i wrote a page that IP bans anything that visits it, and l also put it as a not allowed spot in the robots.txt file.

    This is fuckin GENIUS

    • Moonrise2473@feddit.it
      link
      fedilink
      arrow-up
      8
      ·
      3 months ago

      only if you don’t want any visits except from yourself, because this removes your site from any search engine

      should write a “disallow: /juicy-content” and then block anything that tries to access that page (only bad bots would follow that path)

        • Moonrise2473@feddit.it
          link
          fedilink
          arrow-up
          3
          ·
          3 months ago

          Oops. As a non-native English speaker I misunderstood what he meant. I understood wrongly that he set the server to ban everything that asked for robots.txt

          • Zoop@beehaw.org
            link
            fedilink
            arrow-up
            2
            ·
            3 months ago

            Just in case it makes you feel any better: I’m a native English speaker who always aced the reading comprehension tests back in school, and I read it the exact same way. Lol! I’m glad I wasn’t the only one. :)

      • mozz@mbin.grits.dev
        link
        fedilink
        arrow-up
        5
        ·
        3 months ago

        You need to read again the thing that was described, more carefully. Imagine for example that by “a page,” the person means a page called /juicy-content or something.