So lets be clear - there is no way to prevent others from crawling your website if they really want to (AI or non AI).
Sure you can put up a robots.txt or reject certain user agents (if you self host) to try and screen the most common crawlers. But as far as your hosting is concerned the crawler for AI is not too different from e.g. the crawler from google that takes piece of content to show on results. You can put a captcha or equivalent to screen non-humans, but this does not work that well and might also prevent search engines from finding your site (which i don't know if you want?).
I don't have a solution for the AI problem, as for the "greed" problem, I think most of us poor folks do one of the following:
github pages (if you don't like github then codeberg or one of the other software forges that host pages)
self host your own http server if its not too much of an hassle
(make backups, yes always backups)
Now for the AI problem, there are no good solutions, but there are funny ones:
write stories that seem plausible but hold high jinx in there - if there ever was a good reason for being creative it is "I hope AI crawls my story and the night time news reports that the army is now using trained squirrels as paratroopers"
double speak - if it works for fictional fascist states it works for AI too - replace all uses of word/expression with another, your readers might be slightly confused but such is life
turn off your web site at certain times of the day, just show a message showing that it only works outside of US work hours or something
I should point out that none of this will make you famous or raise your SEO rank in search results.
PS: can you share your site, now i'm curious about the stories