1. Always update to the latest security release
This does not mean that you must constantly update to every new Joomla version that comes out. If you use Joomla 1.5 and you are happy with it, don't update to Joomla 2.5 just because 2.5 is out.
You should always assess the situation and see what the new version will offer you. If there is not any really important new feature for you, DON'T UPDATE.
However, you should ALWAYS update to the latest security releases as these will fix security holes in your system.
2. Hide your administrator login area
The administrator login area can easily be compromised. In Joomla extensions directory there are many plugins that can help you to hide the administrator login area from uninvited guests and hackers.
3. Use the latest PHP version
If you are on a shared hosting, you can't change the PHP version by yourself. However you can ask your hosting provider to use the latest PHP version.
If you don't have a hosting provider yet, always select one that uses the latest PHP version.
4. Use only the extensions you need
Joomla has an abundance of extensions in the Joomla Extensions Directory. Although most of them conform to the JED rules, there are many extensions that are poorly coded. Some of these may create security holes in your system while others might break your Joomla installation.
Always test a new extension on a test environment first before you install the extension on your live site.
5. Use secure usernames and passwords
Don't use the default admin username. A safe password contains at least 8 characters and includes both letters, numbers and special characters.
6. Configure .htaccess
The .htaccess file is a very important tool that can greatly advance your website security. Read this tutorial for more information on .htaccess security.
7. Configure the php.ini file
Read this tutorial to learn how to make your website more secure with the php.ini file.
8. Fix files and folders permissions
Make sure that all files are CHMOD to 644 and directories to 755. You should never CHMOD any files or directories to 777.