There are several different ways to do this.
You can purchase flash drives that have built-in password or PIN protection. They tend to be pricey, but they’re almost perfect for this kind of situation.
Frankly, I don’t recommend them. There are other solutions that are more flexible and less costly.
Become a Patron of Ask Leo! and go ad-free!
Encrypt the disk
One way or another, encryption is the answer.
Whole-disk encryption is an approach that encrypts the entire disk drive, be it a USB thumbdrive or an internal hard disk.
Windows Pro includes BitLocker, which can be used to encrypt an entire drive with a password.
Another approach is to use a tool like VeraCrypt to perform whole-disk encryption. The advantage is that VeraCrypt is not limited to Windows, so in theory your drive would be portable to other systems.
How Do I Encrypt a Disk? has step-by-step instructions for both BitLocker and VeraCrypt.
Encrypt the file(s)
Rather than encrypting and password-protecting the entire disk, another approach is to encrypt only the files containing the sensitive information.
The most common way to do this is to use a zipping utility, like 7-Zip, WinZip, or Windows’ own built-in zip file support. Zip files can be password-protected to encrypt their contents (though not the list of files within the zip file).
The downside is that in order to use the contents of the file, you need to manually unzip it to create an unencrypted copy of the file. You then need to manually re-encrypt it any time you make changes to its contents.
It’s also easy to accidentally leave that copy around, or leave traces of it even after you delete it; but it’s a conceptually easy approach using commonly available tools that you can find across almost all platforms.
Use a dedicated tool
You mentioned that you wanted to save your “websites and passwords”.
Honestly, this sounds like a job for a password vault.
There are several. I happen to use and recommend LastPass, which securely synchronizes your information across all the computers you install it on as well as its own online copy.
However, if online isn’t your thing, there are tools like KeePass, which are designed to store your information in a local encrypted database. You could store it on a USB thumbdrive.
If that’s really what you’re planning to keep track of, a tool like KeePass sounds like exactly what you’re looking for.