So I can now confirm with some data in hand, that if spending a lot of time in your garrison messing around with all the fun stuff on offer there, it will significantly slow your leveling time. My warlock finally managed to hit 100 on Friday, about a week after the expansion release, with a total leveling time of 24 hours and 30 minutes.
I will note that that is still significantly better than my first character leveling times in both Mists and Cataclysm, but it is more than double some of the better leveling times that I've been hearing about from others where a 10-12 hours speed run seems to be about average. Some of that time was the result of server issues in the first week, but a lot of that is just a result of the distraction provided by my Garrison.
I've now started the process on my death knight, and the plan is to run him straight through as quickly as possible, so I'll report back on what the next few toons look like.
In the meantime, on my main, I went off and grabbed the Silver Proving Grounds achievement and started on the legendary quest line. So I'm working on the daily quest for Apexis Shards and running normals to try to get the ilvl to queue for heroics. Once I turn in the Apexis shard portion of the legendary questline, I'll start saving those up for gear next.
I've also managed to upgrade my Garrison to level 3 and am just working on some of the other buildings at this point. It's mostly just a matter of saving up enough Garrison resources at this point to get the other buildings in place. Once that is done, I'll probably start follower missions in earnest.
The current plan for Joar's garrison is
Tailoring - matches a profession
Enchanting - matches a profession
Lumber Mill - extra Garrison resources
Trading Post - extra resources
Workshop - access to extra bonus roll token
I'm also working through Hexweave Cloth as much as possible and am getting pretty close to having enough to craft an epic, so that should help with my ilvl. Enchants are selling well, as are glyphs, so that's keeping my cash flow up pretty well.
My guild has a Molten Core run scheduled for early December, so I'm probably going to wait and do that as a guild run rather than just trying it at random.