If quick format takes a long time, there has to be something wrong, I’d say. Perhaps you are using a very bad connection, or there are a lot of errors occurring.
I’m not sure about what Windows does exactly, but I can imagine that it first makes a list of all files (e.g. to display stats or check if all is ok), while your program probably only has to keep track of the current directory and iterate over the files in there (possibly even without first getting them all into a list or something). If there are a lot of files, I wouldn’t be surprised if this makes a big difference. If your system is low on memory, it perhaps even causes Windows to store so much that it starts swapping memory like crazy.