Well it's out, been out for a day or two now with some great new additions such as the face recognition and near mode skeletal tracking. I've not had any time to really play with it and my ideas but I did discover something daft that I should have realised earlier.
Looking at the green screen sample my brain went "hold on, how is it showing the parts that make up the person instead of just the skeleton?" and dove into the code.
Yes, depth data has all the parts of a person. The original SDK had this as well in the colouring of the players depth data. Why didn't I realise this!
So my trials of bounding boxes, while not a total loss, were based on the assumption I'd have to find all the depth points that make up a person myself.
MS also included an excellent PDF link called Human_Interface_Guidelines_v1.5.0. It covers a lot of points from hardware spec, UI design, usability and essentially says "Hey, we did a whole bunch of research with a whole bunch of people, these are the things you should know" which is really nice of MSR (one of my favourite groups out there).
On the Lync front, I've completed the code for my latest project barring some EWS calls. The ideas surrounding it are simple enough but should hopefully revolutionise how we collaborate and "use space" once I get some Kinect integrated. Small steps (and lots of testing).
Sunday, May 27, 2012
Thursday, May 17, 2012
UCMA 3.0 and presence samples
Following samples is a good way to start things, unless they just don't work for your configured environment. I recently had issues connecting an application endpoint to our Lync 2010 setup, these started with a simple timeout from the server.
Checking a bit more (having confirmed the setup of the trusted endpoint) showed that, unlike the samples I'd found, using the proxy settings for the ApplicationEndpointSettings was a bad thing.
With that out the way, the next hurdle became a mutual TLS exception when using the provisioned settings. All signs point to "you messed up the cert somewhere", but quickly switching over to ServerPlatformSettings and applying the provided cert got the system up and running.
Finally, tying up a persistent RemotePresenceView and event for PresenceNotificationReceived (Task 3) was the annoying. State change events logged a repeated Subscribing, WaitingForRetry. The notification event never even got to fire and the Lync contact for the endpoint never registered as Available.
As a last shot before trying to start from scratch again, I went back through to the start and changed the ApplicationEndpoint to a UserEndpoint, ran it and everything worked. The Lync client showed my endpoint as being available and my subscription to a contacts presence fired at the right times.
I think the issues I had stemmed from a few possible things:
1: Cert issues - did it get properly set on the Lync server? If this was the issue then not using ServerPlatformSettings may have helped resolve 2.
2: I didn't get the Lync server restarted (someone on the MSDN forums had similar issues with an application endpoint not firing events using ServerPlatformSettings, this was their fix)
And finally was it a lack of understanding the workings of Lync in general on my part?
It's probable, but at the same time, samples are there to walk you through a simple process.
Is our environment odd? I know there are some oddities that have been added to make it work with the bastard nightmare child of a phone system we like to call our "primary" phone system, but these don't touch the standard Lync setup I'm talking to.
At least I now have a serviceable framework for tracking presence that I can use to get a reporting system working on.
Checking a bit more (having confirmed the setup of the trusted endpoint) showed that, unlike the samples I'd found, using the proxy settings for the ApplicationEndpointSettings was a bad thing.
With that out the way, the next hurdle became a mutual TLS exception when using the provisioned settings. All signs point to "you messed up the cert somewhere", but quickly switching over to ServerPlatformSettings and applying the provided cert got the system up and running.
Finally, tying up a persistent RemotePresenceView and event for PresenceNotificationReceived (Task 3) was the annoying. State change events logged a repeated Subscribing, WaitingForRetry. The notification event never even got to fire and the Lync contact for the endpoint never registered as Available.
As a last shot before trying to start from scratch again, I went back through to the start and changed the ApplicationEndpoint to a UserEndpoint, ran it and everything worked. The Lync client showed my endpoint as being available and my subscription to a contacts presence fired at the right times.
I think the issues I had stemmed from a few possible things:
1: Cert issues - did it get properly set on the Lync server? If this was the issue then not using ServerPlatformSettings may have helped resolve 2.
2: I didn't get the Lync server restarted (someone on the MSDN forums had similar issues with an application endpoint not firing events using ServerPlatformSettings, this was their fix)
And finally was it a lack of understanding the workings of Lync in general on my part?
It's probable, but at the same time, samples are there to walk you through a simple process.
Is our environment odd? I know there are some oddities that have been added to make it work with the bastard nightmare child of a phone system we like to call our "primary" phone system, but these don't touch the standard Lync setup I'm talking to.
At least I now have a serviceable framework for tracking presence that I can use to get a reporting system working on.
Wednesday, May 16, 2012
Kinect for Windows
Well; I have run through a number of trials and tests using OpenCV and a webcam looking at my own implementation of the sixthsense style of interaction. While the ways of implementing it are there, I sat down with a co-worker one day and realised, why not buy a Kinect? It has less of the over the shoulder portability of a sixthsense system but more of a presentation/presence potential and MS is about to release 1.5 of the SDK.
The ideas I've come up with are interesting, especially since Kinect for Windows is just a bunch of smarts in a box with a camera, IR sensor and IR laser emitter.
Milestone 1 became:
"While the skeleton points are provided easily, I need to locate the object at the point"
After a nice side track off onto depth camera smoothing (a limited implementation of this project which I kind of deduced myself but it's a nice link regardless) and change only depth frame display, I ended up with a quick and dirty depth and skeleton point based bounding box.
Walking points up, left and right using a quick guess at the range of depths to include (to allow for precision problems, angles etc) it gave me a nice framed area for further depth and image mapping.
A happy coincidence was that, when tracking hands, the open palm has a larger area automatically than a closed or half closed hand due to those parts falling outside the allowed +- Z axis. (Here I wrote "Handy!" then slapped myself for the bad pun).
So, Milestone 1 is completed. It's not great but I can make it better and it's required for two of the ideas I'm going to attempt to implement.
Next up, Milestone 2, depth map storing and comparison along with some image recognition.
I'll probably throw my "fun with Lync 2010" things on here as well since that's been driving me up the wall recently.
The ideas I've come up with are interesting, especially since Kinect for Windows is just a bunch of smarts in a box with a camera, IR sensor and IR laser emitter.
Milestone 1 became:
"While the skeleton points are provided easily, I need to locate the object at the point"
After a nice side track off onto depth camera smoothing (a limited implementation of this project which I kind of deduced myself but it's a nice link regardless) and change only depth frame display, I ended up with a quick and dirty depth and skeleton point based bounding box.
Walking points up, left and right using a quick guess at the range of depths to include (to allow for precision problems, angles etc) it gave me a nice framed area for further depth and image mapping.
A happy coincidence was that, when tracking hands, the open palm has a larger area automatically than a closed or half closed hand due to those parts falling outside the allowed +- Z axis. (Here I wrote "Handy!" then slapped myself for the bad pun).
So, Milestone 1 is completed. It's not great but I can make it better and it's required for two of the ideas I'm going to attempt to implement.
Next up, Milestone 2, depth map storing and comparison along with some image recognition.
I'll probably throw my "fun with Lync 2010" things on here as well since that's been driving me up the wall recently.
Subscribe to:
Posts (Atom)