I'm with Abbysmum.
The old tablet is usually just fine for a first timer, or a kid, or when you want to save a few bucks, there's nothing wrong with it. (sidenote - if you're buying used beware of stolen items. An iDevice can appear as though it is fresh and ready to set up on the "Hello" screen - but if it has been wiped remotely or something, you'll only get to about step 4 before it'll ask you "enter the apple id of the person who wiped this device before you can proceed" - a nice security feature Apple has built in to prevent theft)
Your only drawback is the older versions sometimes can't get the new updates, which means they still work, but some of your favorite apps require a higher operating system than the one you have. I have an old iPod4 that I bought brand new at the time and now can't be updated past iOS 6 I believe. This is fine for me because I just listen to music with it mostly, but I know some of the apps I've wanted to put on it for kids won't even download because they require iOS 7 or higher.
So, in the end, it's up to you. If it's a first time tablet for a kid or something for them to use to show their responsibility, you can go used - just don't go too old. If it's something that will be used a decent amount and you plan on having it for a long while, bite the bullet and buy new. The new hardware will run a lot smoother with the software and you'll be less prone to crashes.
Hope that helps you with your decision. It's a lot of money to spend, but if you think you'll have it a few years, it's not all that bad