Silverlight interoperability

Introduction

By now you’ve probably read a thing or two about Silverlight 2 already. About the layout system, the new set of built-in controls, data-binding, styles, templates, yada yada. You may either already be cranking out code utilizing all of these visually-oriented features, or you may be left wondering:

"How is any of this useful to me? I’m already using HTML/CSS/JS on the client, and I don’t have any intentions to visually enhance or replace my website."

My goal is to demonstrate how you can take advantage of Silverlight by taking you through some of the features in Silverlight that you can use to enhance your existing website in a non-obtrusive way. Think:

1. Writing new code in a managed language (C#, Ruby, JScript.NET, whatever) instead of native (interpreted) JavaScript.
2. Using OpenFileDialog to read files on the client, without round-tripping to the server.
3. Storing transient data securely on the client in isolated storage.
4. Improving responsiveness and performance by executing work in the background through a BackgroundWorker or by using ordinary threads.
5. Accessing cross-domain data via the networking APIs.
6. Retrieving real-time data from the server via sockets.
7. Binding data by re-using WPF's data-binding engine.

If you’re wondering what the point of all this is, ask yourself (or the users of your website) when the last time is you (or they):

1. had no idea how much time there was left before an upload would finish;
2. couldn’t resume from a failed upload;
3. lost data (article/comment/etc.) after trying to save it;
4. wasted lots of server-side bandwidth (and decreased client-side responsiveness) because the server had to act as a proxy between the client and a third-party website.

This can be addressed with Silverlight. The key to integrating all of this is the two-way interoperability layer in Silverlight (sometimes referred to as 'the HTML/JS bridge'), which is what I will be going through in this article. I will show you how you can access the HTML DOM and use your existing JavaScript code. You will see how you can pass managed objects back and forth, and program against them in JavaScript like ordinary JS objects. We will dive deep and take a look at what really happens under the hood to understand how far exactly our interoperability goes.

Getting started

Accessing the HTML DOM

Silverlight’s gateway to the browser is System.Windows.Browser.HtmlPage. It’s a static type with properties – such as Window, Document and Plugin – which should all be self-explanatory. The only thing to note is the IsEnabled property. Access to the hosting page may be disabled in a few situations:

1. The hosting page set the enableHtmlAccess hosting parameter to false. This is a common thing to do when hosting advertisements or other Silverlight content that you don’t trust.
2. Your code executes within a custom host, such as the Blend designer.
3. The plugin unloaded itself, either by removing itself from the HTML DOM, or by changing its source property.

Generally you’ll check for this flag before initializing code that uses HtmlPage, and bail out quickly if it returns false. There’s usually no need to check for this flag throughout the rest of your code.

With this information it should be straightforward to access the HTML DOM. To insert a new element into the DOM, you may previously have written:

1 
2 
3 

var element = document.createElement("div");
element.innerHTML = "Hello, World!";
document.body.appendChild(element);

This translates to the following in C#:

1 
2 
3 

var element = HtmlPage.Document.CreateElement("div");
element.SetAttribute("innerHTML", "Hello, World!");
HtmlPage.Document.Body.AppendChild(element);

You get the idea.

A common thing you may run into is that when the plugin is initializing (the application's constructor executes), the HTML DOM may not be ready yet. This means you may not be able to find an element, or you may run into 'Operation aborted' errors in IE. To deal with this, you are encouraged to check for HtmlPage.Document.IsReady and handle HtmlPage.Document.DocumentReady if the document isn't already ready.

Invoking JavaScript

Calling into JavaScript isn't much more complicated. To continue getting right to the point, let's take the following JavaScript:

1 
2 
3 

var calculator = new Calculator(); // Assume this is a JS library of ours.
var sum = calculator.add(5, 1);
alert(sum);

And turn it into C#:

1 
2 
3 

var calculator = HtmlPage.Window.CreateInstance("Calculator");
var sum = Convert.ToInt32(calculator.Invoke("add", 5, 1));
HtmlPage.Window.Alert(sum.ToString());

There are a couple of things going on here.

The first thing to notice is that you use HtmlWindow.CreateInstance to instantiate JavaScript objects. Under the hood this will evaluate some JavaScript to create the instance, due to lack of native support for instantiating objects through the browser's plugin API.

Next, notice that Window, an HtmlWindow type, has an Invoke method. It inherits this method from one of its base types, ScriptObject. A few other fundamental methods it inherited are InvokeSelf, GetProperty and SetProperty. These few methods let you do just about anything. I will cover them in-depth later in this article. For now all you need to remember is that these members let you get and set values, and invoke functions.

The other thing to notice is the call to Alert. We could've actually called into alert like this:

1 

HtmlPage.Window.Invoke("alert", sum);

In fact, that's exactly what HtmlWindow.Alert does under the hood. In general we decided to make commonly-used functions first-class for discoverability and sometimes performance reasons. Whenever there is a built-in way to do something, you'll want to use that instead of using late-bound calls.

Lastly, Invoke and GetProperty both have an Object return type. They may return a bool, double, string or ScriptObject. We always return a double for numbers because this is what Safari gives us back under the hood. Also, for now assume that ScriptObject is an ordinary JavaScript object.

With this knowledge you should be able to do just about anything from managed code already. You can instantiate objects, invoke functions that return objects, set properties on those objects, and so forth.

At this point you may be wondering what happens when you pass a managed object to JavaScript. Before we go into this, let's quickly go over the opposite of what we've done so far.

Invoking managed code

Before you can invoke any managed code from JavaScript, you first need to register the objects you want to expose to the browser. You also need to mark which members are callable with the ScriptableMemberAttribute, or mark all declared members at once with ScriptableTypeAttribute:

1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 

public class Calculator {

    [ScriptableMember]
    public int Add(int a, int b) {
        return a + b;
    }

    // Not callable from script.
    public int Subtract(int a, int b) {
        return a – b;
    }
}

public class App : Application {

    public App() {
        HtmlPage.RegisterScriptableObject("calc", new Calculator());
    }
}

After you have a scriptable object, you can use it from JavaScript:

1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 

void onPluginLoaded(plugin) {
    var calc = plugin.content.calc;
    // If you are using ASP.NET AJAX to create a Silverlight instance, use this instead:
    // var calc = plugin.get_element().content.calc;
    var sum = calc.Add(5, 1); // Case sensitive.
    alert(sum);
    try {
        calc.Subtract(5, 1); // Failure. There's no such member.
    } catch (e) {
        alert(e.message ? e.message : e);
    }
}

As with calling into JavaScript from managed code, you can pass JavaScript objects and DOM elements to managed code, assuming the signature of the invoked member permits this.

You should make sure you don't access any managed objects before they're initialized. If you need to access them when the page is loaded, you should handle the plugin's onload event and access your managed objects from there.

Inside the marshalling layer

So far the examples we've seen are very basic. They involve calling methods with primitive arguments and return values. In Silverlight our support goes much further however. As I briefly mentioned before, you can pass arbitrary managed objects back and forth between Silverlight and the browser. We also have specialized support for DateTimes, Guids, structs in general, Delegates, lists and dictionaries.

JavaScript dates are ordinary objects. This is why we treat them as such by marshaling them to ScriptObjects, just like we do with any other type of JavaScript object. The only difference is that we check what type of value the target site expects. If it expects a DateTime, we will convert it to that. Similarly, when you pass a DateTime to JavaScript, we will convert it to a real JavaScript date object.

Our support for Guids is trivial. When one is passed to JavaScript, the marshalling layer converts it to a string. We do this because it's fairly common to use Guids as identifiers for data, and the easiest way to represent them in JavaScript is via an ordinary string. If you round-trip such a string back to managed code, we will only attempt to convert it back to a Guid if the target site expects a value of this type.

Struct support is relatively basic too. We try to enforce the right semantics by making sure we create copies of values before marshalling them, which involves boxing the value. Without explicitly copying the values, there are cases where a value type would very much act like a reference type. For example, passing a value type to JavaScript where one of the fields gets mutated would in fact mutate the field on the value on the caller's site.

Anything else that is not a ScriptObject or primitive will be marshaled by reference as a managed object. To understand how managed objects are marshaled, we should first take a look at how browser objects are marshaled.

Browser object marshalling

All browser objects are represented by ScriptObjects. This includes ordinary JavaScript objects, but also windows (HtmlWindow), documents (HtmlDocument) elements (HtmlElement), element collections (HtmlElementCollection) and even managed objects (ManagedObject, an internal type) that have gotten a life inside the browser. We do a lot of things when the browser returns an object reference to us as after invoking a function or property. This is all done transparently, so you don't need to worry about any of this, but it gives you an idea of what's really happening in part of the marshalling layer.

Type inference

In Silverlight 1.1 alpha we forced you to tell us what the type of an object was going to be via a generic type parameter. While the glorified cast may seem nicer than the casts you need now, you would quickly find that you don't always know the type of an object. In one place you may assume the elements of an array are of type ScriptObject, while in reality one of the elements is in fact an HTML DOM element. This made it difficult for us to guarantee correctness, impacting the reliability of any code that used our bridge.

This has changed in Silverlight 2. We now automatically determine the type of an object. In Internet Explorer we can do this very efficiently through a few native calls to QueryInterface. In the other browsers we are forced to determine the type via some 'JavaScript reflection.' In both Firefox and Safari this boils down to inspecting nodeType to see if an object is a document or element. If the object doesn't have this property, we will check if the object is an instance of HTMLWindow, NodeList or HTMLCollection in Firefox. In Safari 2 we'll look at a few other properties that let us determine whether it's a window or element collection.

We only need to do this when the value that the browser returned to us is an object. For every value the browser returns to us, it lets us know whether it's a bool, int, double, or an object, so we don't always need to do additional work to find out the type of a return value.

Object identity

When the browser returns a value, we convert or marshal this to a managed value. For primitives this is a straightforward process. Objects however are simply a pointer, so we need to marshal it to something that is usable from managed code. We need to marshal it to a ScriptObject or one of its derivates, based on the type we previously inferred.

The simplest thing we could've done is marshal a pointer to a new object every time. This would have several consequences though. In scenarios involving lots of round-tripping of objects, memory usage would grow and the pressure on the GC would increase. More importantly, we would have a correctness problem. Imagine the following code:

1 
2 
3 
4 

var f1 = HtmlPage.Document.GetElementById("foo");
var f2 = Htmlpage.Document.GetElementById("foo");
assert(f1.Id == f2.Id); // True.
assert(f1 == f2); // False.

This code queries for the same element twice. The variables f1 and f2 both refer to a different managed object, which in turn both wrap the same native browser object. This object identity issue might seem unimportant, but it hits you whenever you need to tell whether two objects are the same.

We could deal with this by overriding Equals/op_Equals and letting it compare the underlying browser object pointers, and let GetHashCode return the pointer's hash code. Although this would probably address the majority of the situations, it would simply be a work around. There still wouldn't be true object identity. You would still get the wrong result when comparing two variables typed as Object, as the CLR would end up comparing the references, rather than calling into the overloaded equals operator. Furthermore the GetHashCode implementation would be rather inefficient on Safari – which I'll explain further in a second.

To correctly implement object identity, we need to know whether we already marshaled a browser object before. If we did, we simply return that object. Otherwise, we marshal it and make sure we keep track of the managed object we create. We introduced an object cache in ScriptObject for exactly this.

The object cache maps pointers to WeakReferences of ScriptObjects. By using weak references we avoid keeping alive objects forever. Additionally, ScriptObject implements a finalizer to remove its entry in the object cache, such that we're not leaking at least sizeof(IntPtr) bytes per object that no longer exists.

At this point there's only one remaining problem. Safari doesn't preserve object identity in its plugin API, which means that we get back different pointers every time for the same native browser object. It is important to note that the pointers we get back are pointers to objects which wrap the real browser object – even so in FF. Both browsers support the NPAPI, a plugin API supported by most browsers, which defines the contract of an object (NPObject). To adhere to this contract, it usually needs to marshal its browser objects itself. Firefox seemingly does this by caching NPObjects to preserve object identity and reduce memory overhead. Safari apparently creates a new NPObject every time.

We addressed this problem by introducing a second object cache, which maps a custom identifier (an int) to an IntPtr of the browser object that we marshaled before, which is also the key in the primary object cache. We generate custom identifiers ourselves, and try to store this in a private field ($__slid) on the object, such that we can read this back later when we need to marshal another pointer. Whenever we are trying to marshal a pointer, we first try to read this private field and do a lookup in our secondary object cache to get back a different pointer (to this object) which we marshaled before.

This secondary cache is purely used as a heuristic to improve performance. We don't rely on any of the information we gather in this process. We use it to do a lookup in the real object cache, and we then call into script to verify the two pointers we have (a pointer which we want to marshal, and a pointer we marshaled before and believe points to the same browser object) do indeed point to the same browser object. We do this by calling into an anonymous JavaScript helper function, which looks like:

1 
2 
3 

function(obj1, obj2) {
    return (obj1 == obj2);
}

When we pass two pointers to native browser objects, NPObjects, the browser will marshal these back to real browser objects. In every browser, including Safari, it will get back the original browser objects. Because of this, we can correctly tell whether two objects are the same in JavaScript.

Back in managed code we can use this to verify that two pointers are indeed the same. If they are, we go ahead and return the managed object we created before for this object. If they aren't, we will walk through our primary object cache and check if this pointer really wasn't marshaled before, by comparing each pointer to the current pointer via a call to our JavaScript helper function. If it turns out we never marshaled this browser object before, we will generate a new identifier and associate try to associate it with this browser object. We will update both object caches accordingly.

This process does involve a slight performance hit on Safari, in particular when you often marshal objects that you never marshaled before. Unfortunately there's not much we can do about this given the way Safari decided to implement the NPAPI, without sacrificing the usability of our managed API.

Managed object marshalling

Marshalling to JavaScript objects involves creating a browser object which wraps the managed object and acts as a proxy. Internally it's a ManagedObject which derives from Scriptobject. When a managed object is round-tripped, we will return its ManagedObject, rather than the real managed object it wraps. You can access the underlying managed object via the ScriptObject.ManagedObject property.

We decided to return ManagedObjects rather than their underlying objects to make the programming model more consistent and increase the reliability of user code: every object can be treated in the same way, just like you can in JavaScript. This means you don't have to constantly check if an object is either a browser object or a managed object.

ManagedObjects reference a ManagedObjectInfo which stores a method table with entries for each scriptable member for that type. The entries are wrappers around the actual members, which makes it possible for us to add special-casing for certain type of members, such as properties and events.

For every managed type we lazily construct the (case-sensitive) method table and cache this information for performance reasons. We use weak references such that this information can be released when there's no reason to hold onto it any longer. The process for constructing the method table for most types involves walking through all public members with a ScriptableMemberAttribute. We then walk their inheritance tree. For every type in this tree with a ScriptableTypeAttribute we add all of that type's declared members. This means that if you apply this attribute to your type, it will only add the members you declared in your type; it does not include any inherited members. We also add an addEventListener and removeEventListener method to the method table to support events in the same kind of way they're supported in the HTML DOM.

Another method we add is createManagedObject. This method lets you create instances of complex types that are used as parameters for any of the scriptable members. For example, if you have a CustomerService type with a scriptable Add(Customer) method, you will be able to create an instance of such a Customer type, assuming it has a public default constructor:

1 
2 
3 
4 

var customerService = plugin.content.customerService;
var customer = customerService.createManagedObject("Customer");
customer.Name = "John Doe"; // Assume we marked Customer.Name as a scriptable member.
customerService.Add(customer);

By automatically doing this work for you, you don't need to add calls to HtmlPage.RegisterCreatableType all over the place. You also don't need to hold on to the plugin object in order to call createObject to instantiate a type registered via RegisterCreatableType. In other words, each object that is a receiver (i.e. has a scriptable method which takes a complex object) is also a factory which creates complex objects.

We have basic support for method overloading, by finding the overload which matches the number of arguments and by doing some basic parameter validation.

Delegates are a special-case scenario. We map a delegate's DynamicInvoke method and add an apply method. This method allows you to invoke delegates in a 'late-bound' fashion from JavaScript. We will take care of unrolling the array of arguments and call DynamicInvoke on the delegate.

JavaScript functions are automatically marshaled to delegates if the target site expects one. Silverlight 2 currently only supports delegates that are compatible with the void(object, EventArgs) signature, such as EventHandler and EventHandler<TEventArgs>. Arbitrary delegate types aren't supported – at least not at the time of writing.

Types that implement IList will end up storing members in the method table that correspond to the functions that JavaScript arrays support. This means managed arrays, List<T>s, and so on can be treated and used as arrays in JavaScript.

Similarly, types with support for IDictionary<string, TValue> are also made more accessible from JavaScript. The keys in such a dictionary translate to fields in JavaScript. When you set a field, we will add or set the value with the field's name as the key. Accessing a field translates to a lookup in the dictionary.

Lastly, we always map the ToString method of every type, because some browsers implicitly call toString on objects, for example when passing them to alert.

Object lifetime

Normally a managed object gets collected by the GC when it's no longer rooted. In other words, when an object can no longer be accessed, the GC is allowed to collect it. When we pass such an object to the browser, we have to prevent this behavior. We do this by pinning the ManagedObject and giving lifetime ownership to the browser object that wraps it. This browser object is always a reference counted object which will unpin the ManagedObject when the last reference is released.

We will re-create the native browser object on the fly when a ManagedObject is passed to the browser after its native counter-part no longer exists.

If the managed runtime shuts down when the browser still has a reference to a managed object, for example by removing the plugin from the DOM after getting the reference, the browser object wrapping the managed object will know about this and gracefully let all calls fail. When the last reference goes away, it won't attempt to unpin the managed object either, which isn't a problem as the managed object was already freed from memory during the managed runtime shutdown.

The lifetime of all other objects is more trivial. Either when the managed runtime shuts down, or when a ScriptObject gets GC'ed because it's no longer reachable, we will release our reference to the native object. We guarantee that this reference is released on the UI thread.

Exception handling

Silverlight returns exceptions thrown in managed code as JavaScript errors. It does this by calling ToString on the exceptions, and then it passes this exception text on to the caller. Internet Explorer stores this in an error object. When catching this error, you can get back the exception text by accessing the message field. In all other browsers the object that you catch is the exception text.

There's one exception. It appears that Safari ignores any reported errors. This means that if you invoke a method which throws, your JavaScript code won't know if anything went wrong. It'll simply not get back a value. One way to deal with this is by always returning something from managed code, such as a Boolean value. This way you can assume something went wrong when you don't get back any value.

The opposite – throwing exceptions in JavaScript back to managed code – works similarly. Unfortunately the plugin API we use for browsers other than Internet Explorer (NPAPI) doesn't have first-class support for reporting exceptions. We can only tell whether a call failed or succeeded. We considered invoking JavaScript functions via a helper function, but this requires the ability to invoke JavaScript functions in a late-bound way (via Function.apply), something that isn't supported by every version of Safari that we support. Consequently we simply throw a generic InvalidOperationException whenever a JavaScript function invocation fails. In Internet Explorer it contains the error text of the exception that was thrown, while in all other browsers you will simply see a generic error message.

Use case: Extending the browser with OpenFileDialog support

Most browsers today don't support reading files on the client. You have to use an input element of type 'file' and let the browser send the file to the server, which can then echo the contents back to the client. Besides the somewhat awkward user experience, it also complicates the programming model. You have to restore the state of the page, or you have to use an iframe and do the round-tripping there.

Silverlight comes with an OpenFileDialog feature. This dialog provides read-only streams to one or more files selected by the user. Out of the box you can only use this feature from managed code. It takes little effort to encapsulate this feature though. Let's take a look at what it takes to extend the browser with this feature.

Step 1: Adding a Silverlight island to the page

There are several ways to do this. The easiest is to use VS and let it generate a page with Silverlight for you. It will embed Silverlight via an object tag, which refers to your Silverlight application (a .xap file). To integrate this application with your website, all you need to do is copy this object tag declaration and the .xap file.

Since our feature doesn't have any UI on the page, you can make sure the object tag's width and height is 0 pixels. You shouldn't use the display or visibility styles to make Silverlight invisible, as Silverlight may not even run at all in that case.

Step 2: Making OpenFileDialog scriptable

We can make OpenFileDialog scriptable by introducing a few wrappers. For the OpenFileDialog type itself we can simply add a scriptable method to our application. We'll let this method return an array of scriptable objects which wrap the OpenFileDialog's SelectedFiles property. In code this looks like:

1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 

public class App : Application {

    public ScriptableFileInfo OpenFiles(string filter) {
        var dialog = new OpenFileDialog();
        dialog.Filter = filter;
        if (dialog.ShowDialog() == DialogResult.OK) {
            var fileInfos = new ScriptableFileInfo[dialog.SelectedFiles.Length];
            for (int i = 0; i < dialog.SelectedFiles.Length; i++) {
                fileInfos[i] = new ScriptableFileInfo(fileInfos[i]);
            }
        }
        return new ScriptableFileInfo[0];
    }
}

And the ScriptableFileInfo type can be implemented like this:

1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 

public class ScriptableFileInfo {

    private FileDialogFileInfo _info;

    [ScriptableMember(ScriptAlias = "contents")]
    public string Contents {
        get {
            using (var reader = new StreamReader(_info.OpenRead()) {
                return reader.ReadToEnd();
            }
        }
    }

    [ScriptableMember(ScriptAlias = "fileName")]
    public string FileName {
        get {
            return _info.FileName;
        }
    }

    public ScriptableFileInfo(FileDialogFileInfo info) {
        _info = info;
    }
}

Now, we still need to expose this to script. There are two options we can choose from:

1. Add a ScriptableMemberAttribute to our OpenFiles method, and register our application object as a scriptable object by calling HtmlPage.RegisterScriptableObject.
2. Encapsulate our OpenFiles method via a delegate which is accessible from script.

We will go with the last option because it's the easiest to work with from JavaScript. It's also just as easy to implement. In our application's constructor we simply add:

1 

HtmlPage.Window.SetProperty("openFiles", new Func<string, ScriptableFileInfo[]>(OpenFiles));

Step 3: Using OpenFileDialog from JavaScript

Now that we've added an openFiles function to the window object, we can access it from JavaScript like this:

1 
2 
3 
4 

var selectedFiles = window.openFiles("Text Files|*.txt;*.csv|All Files|*.*");
for (var i = 0; i < selectedFiles.length; i++) {
    alert(selectedFiles[i].fileName + ": " + selectedFiles[i].contents);
}

We do still need to be careful about when we execute this code. The openFiles function won't be there until our application has been downloaded and its constructor has executed. You can account for this by handling the onload event on the Silverlight plugin. Once this event has occurred, you should be able to safely use the openFiles function on the window object.

Conclusion

Silverlight does a lot of work under the hood to make it a first-class citizen in the browser. People writing managed code can fully integrate with the existing codebase for a website. Those who prefer to continue writing JavaScript can fully leverage the features offered by Silverlight with little effort. This should allow everyone to get at least something out of Silverlight.