This article goes for windows XP sp3 (i checked it on sp3 only) and windowws 7.

in order to change the taskbar preview delay follow this:

1. Click ‘start’ button < type ‘regedit’ < press ‘enter’
2. Browse your registry to HKEY_CURRENT_USER\ Software\ Microsoft\ Windows\ CurrentVersion\Explorer\Advanced
3. Right click Advanced<click’ New’ <click ‘DWORD (32-bit) Value’
4. Name the DWORD value ‘ExtendedUIHoverTime’.
5. Right click the ‘ExtendedUIHoverTime’ value<click ‘modify’.
6. Enter the requested value< click ‘ok’ (default value is
400)
7. Reset the computer to view the new settings.