Re: An opes usage question.

[Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx>]

On Thu, 8 Jan 2004, John G. Waclawsky wrote:

> Alex, thank you for your reply and the pointer. I agree with you
> that the use case I am interested in is not explicitly documented in
> the opes material. This gives the wrong impression, since all the
> examples show the data flows returning to the first hop data
> dispatcher. To restrict data flow in this way would be inefficient
> for high capacity applications and would tend to make the first hop
> a bottleneck.

John,

	You are misreading OPES architecture diagrams because you are
thinking about a much "simpler" case than most diagrams attempt to
document and, hence, you assign wrong roles to "boxes" that may not
even exist in your environment. This is the architecture draft fault,
not yours. Let me try to explain.

Here is a "classic" OPES proxying case (horizontal lines are
application data/protocols being proxied, proxies may be adapting data
internally):

    Figure A.

        end -- proxy -- proxy -- ... -- end

Here is a "classic" OPES callout case (vertical lines are callout
data/protocols, proxy is not adapting data, callout servers may be
adapting data):

    Figure B.

        end --- proxy --- end
         _________|__________
         |        |         |
         |        |         |
      callout   callout   callout
      server    server    server

A combination of the above is possible and common, of course. In a
combined case, some proxies are adapting data internally and some use
callout servers to adapt.

In your particular case (the adapted data flows to the next hop), you
do not have callout servers. You have proxies that adapt data
internally. However, your case may not be a classic proxying case
because you may want proxies to use a protocol that differs from the
original application protocol (so that you can ship metadata and
perhaps pipeline more efficiently). I will use curly (~) lines to show
that new protocol below. You may also want to do some load balancing:

    Figure C.

                    ~~~~~ proxy1 ---
                   /
      end -- proxy ~~~~~~ proxy2 ---  ??  end
                   \
                    ~~~~~ proxyN ---


I do not know how the proxies will get application data to the right
"end", but it is not important for this discussion.

Note that the curly path may use a completely new protocol (e.g., OCP)
or can use a combination of the original application protocol to
deliver data and some other protocol for metadata (billing, etc.)
records.

The above is OPES, but not a callout case. There are no callout
servers, just proxies. Whether you want to use OCP for the curly path
depends on what kind of data/metadata you want to exchange and what
other protocols you have available.

> To be a little more specific on my use case and to make sure I
> understand your response let me add more detail. Fundamentally, I wish
> to have an "open" IETF network environment. The second requirement is
> speed in adaptation processing, so I am thinking that callout will be
> faster than using a proxy (maybe this is an incorrect assumption).

It's an incorrect question :-). Adaptations at the callout server can
be as "fast" or as "slow" as adaptations performed internally at the
proxy. The primary reason to use a callout server is to "outsource"
(from business, development, support, logistics, legal, etc. points of
view) adaptations. The primary reasons not to use a callout server are
security and overheads of the data having to return back to the proxy.

> I may be a little confused about the distinction between classic
> proxy and classic call out in this situation (this word proxy isn't
> even mentioned in the opes architecture document as a
> consideration).

I hope the above diagrams clarify the distinction. OPES processor is
the term closest to the "proxy", I think.

> Also, I have a requirement to adapt a large volume of content for
> numerous devices.

Both callout and builtin adaptations can handle large volumes of
content. There are many factors at play here, from legal concerns to
the kind of adaptation being performed.

> I expect that the volume of data will require a number of adaptation
> (call out) processors. I estimate 10 of them and therefore I would
> need load balancing as part of the solution for directing data to
> the call out servers (doing the adaptation).

Load balancing can be done with both callout and proxying schemes.
OPES mechanisms are usually per-message so any load balancing method
that does not split individual application messages should work just
fine.

> I am looking at Figure 3 in the opes architecture draft and
> considering a load balanced collection of ten call out servers. As I
> had mentioned previously, I'd like to have parallel pipeline
> approach where the adapted data from each of the opes call out
> server is simply forwarded on, directly to the producer or data
> consumer (what ever the case would be).

The latter makes the adapter a proxy, not a callout server (by
definition). See Figure C above.

> In addition each of the opes call out servers would then send
> billing and trace data back to the load balancer (or another network
> location). Is it realistic to expect that this could this be
> accomplished within the opes framework?

It is. The only question is what protocol(s) you will use between your
proxies (the curly lines in Figure C).

> In fact a more fundamental question is the opes framework the best
> way to solve this problem and maintain an open system. I think this
> is what opes is all about.

IMO, OPES framework accommodates, in principle, any intermediary
adaptation. Thus, your problem is within the framework. However, the
framework is not complete. It does not have ready-to-use tools for all
possible intermediary adaptations. I hope that you will find OCP Core
and OPES communications "tools" useful for your problems, but I lack
information to tell your whether they are the best.

The system is "open" if it uses "open standards". If current OPES
tools are not right for you, and there are no better open tools, then
we can work together, within the OPES Framework, on defining open
standards (protocols and interfaces) that will solve your specific
problem.

Alex.