Building a great touch UI can often be more difficult than a traditional desktop UI due, in part, to the lack of input accelerators like right-click, scroll wheel, or keyboard shortcuts.
One way to approach this challenge is to focus initially on a great touch-oriented UI. You can still do most of your testing using the desktop target for its iteration speed. But, remember to switch frequently to a mobile device to verify that everything feels right.
After you have the touch interface polished, you can tweak the visual density for mouse users, and then layer on all the additional inputs. Approach these other inputs as accelerator—alternatives that make a task faster. The important thing to consider is what a user expects when using a particular input device, and work to reflect that in your app.