EThOSnet Home

EThOS Toolkit (guidelines & further information available here)

Resources

News

Contact Us

Who's Involved?

Existing Arrangements

The Vision

The 7 Strands

Project Documentation

Theses Digitisation Project

Old EThOS website

Questions and Notes on Web Services Orchestration Presentation

Q: How many staff do you have to maintain this fantastic but highly complex system?
A: Myself and a very talented developer called Simon Lamb.
Q: Without trying to sound evil, if he slips under a bus, where are you?
A: In deep trouble at the moment. The unit within which we work has been changed. As part of that change, it is very, very high on our priority list to get someone else to be involved with this so that if and when Simon goes or something happens to him, we’re covered.
A: I have to add that the under a bus scenario applies to the whole university, not just in this case.
A: The other aspect is that we have done this all on JISC funding and JISC funding pays for one programmer and not several who can share the load.

Q: As you have several Web Services involved in that across the engine, who is responsible for managing the transactions?
A: In what sense?
Q: In the sense that, if the last image can’t be ingested, can you roll back the whole thing?
A: The BPEL engine we use has got a rollback facility.
Q: So it is in the orchestration engine room?
A: Yes.

Q: This might be a bit tangential, but have you got any thoughts on how this works in an environment where you might have content that you can’t for various reasons send to Web Services that are in certain countries or if you’re doing text extraction on content that is private or sensitive?
A: No, in a word. I have no view on that at all as it is not something that we have seriously considered. The short and glib answer would be that if it is sensitive in that way, you do it in-house if you can.
Q: But would the engine itself be able to tell? If for example, I do animal research, I want to keep my content stable and within my university and I don’t necessarily want my name and address going through a third party service.
A: If you can programme that decision by saying ‘ depending on the metadata that comes with these objects, then we need to be careful with it’ then yes it will. It may be that your preferred choice is to send it off to somewhere in Nigeria and that is normally fine, but on this particular occasion it is sensitive for whatever reason – would it therefore be ok to send it to the States although this will make it a slower Web Service? Yes as long as you have the appropriate checks, yes you can.

Q: Certainly one instance of BPEL is the Oracle one. Their instance, if you can afford it, has an associate of rules database and the idea is that the BPEL workflows actually adhere to those rules and you can put in your own rules. So you can imagine, for instance, applying a rule which says that if this content is of a certain type then the rule will determine that available Web Services for it are limited to these, instead of this wider bunch.
A: Following on from that, you can build stages into the orchestration and say point at this Web Service and if you don’t get a response within 2 minutes - it’s down – and go throw it at that one instead.

Q: Given that all your instances are at Hull and most of your Web Services are also there; if you were sitting down and doing this again from scratch, would this be the way that you would do it or would you say ‘ I wouldn’t use Web Services at all’? I’m just curious to know.
A: Well effectively we have had to answer that question because with the Hydra project we’re taking all the things we learned in REMAP etc. but effectively starting again. The answer is yes and no. We are not going to go down the SOAP road again, we are going to do Hydra with REST. But that has only been a solution available to us for the last three weeks because the whole REST Muradora interface has only been available for that period of time. There was a REST interface that came out with 3.0 but that came with a few key components missing.
Q: I’m just trying to make a judgement in one way for whether Web Services within an application repository as opposed to something like DSpace or EPrints which is exposing Web Services at the edges is in itself a good or bas thing - I have no idea.

Q: Hold your thoughts in your mind because one thing  I’d like to attend to in the discussion is having heard about Web Services and in many cases simply being picked up because they were there; on reflection if we want to do carry out a lot of these functions – are Web Services the route to actually go down and what alternatives are there that we might consider?

Q: I was wondering if the deposit tool and the BPEL script was open source?
A: Yes is the simple answer, but it’s not that simple because we’re talking about Web Services. You’d have to run local or different Web Services that YOU wanted to implement. I can’t give you a CD or DVD and say stick this in the machine and it’s going to work – there’s just no way. If on the other hand, if you want sight of the BPEL scripts so that you’ve got a starting point to build on – then yes.

Q: Do you find there is any risk built in with using a whole assortment of Web Services hosted elsewhere, in say, those services changing in such a way that they no longer fit with your system, or perhaps you find services that are being dropped?
A: Potentially there is, which is why we’re running most of those services in-house even if we didn’t actually need to. That is one of the pieces of logic. We then know what is going on. In the case of something like PRONOM where it is their nature to change and you actually want to take advantage of that, then, fine – go outside your institution. There’s no reason why we couldn’t have used DROID and the national archives at the National Archives other than installing it locally in the whole. The only reason we didn’t was that we could see that to potentially get the DROID signatures, to get the information about what files we had, you have to transfer that entire file – no matter how big it might be – across the web to the National Archive and then get a signature back. It struck us that it was far easier to put a local instance of it and not do the transfer. But that was a decision we made.

Q: This is another area we must re-visit in the discussion. Web Services have quite a lot of history and go back a good few years and yet we’re still sitting in a room having a discussion about whether they’re a good or a bad thing to use. So you could say that the case for the use of Web Services is not clearly proven. But the reason for that is that Web Services rely on the two partners communicating to each other being aware and promising to deliver a service to each other - being there when they call. It took an awful long time for those bilateral agreements to become a little bit stable. But clearly people are starting to move down the line that such a model is feasible and to take one example of one that, the BL is heavily involved in is the Planets project (http://www.planets-project.eu/) along with a number of other European partners which is seeking to provide a whole host of different preservation services which it is going to be delivering as Web Services – at least as one of the options; because they believe, from their perspective that they are proving them in the most convenient way and embedding them in other workflows as part of other repository instances. They are confident that they will have that other half of the equation there waiting for them – whether that is the case or not, I don’t know.