"memory wipe sounds great. But if you mark the location on a map or write it in a diary you will still have that after the memory wipe. I had the idea to memory wipe the "bad people" in my story but not sure how that will go."
That's the thing. You could easily use that as a plot device - a villain who is in the habit of keeping diary finds the place, but gets memory wiped... but because he has his diary, he can gather all necessary information and later abuse the tower's power, having found a way to circumvent the memory wipe system.