0 Comments

Apache Camel's YAML DSL examples are currently in some parts lacking when compared to Java and XML samples. One thing I was trying to figure out was how to configure error handler using YAML.

Here's an example for future reference:

- error-handler:
    dead-letter-channel:
      dead-letter-uri: "https://webhook.site/b7562774-4939-4e90-9337-5b9dd4d2ff1d"
      redelivery-policy:
        maximum-redeliveries: 3
- route:
    from:
      uri: timer:timer
      steps:
        - setBody:
            expression:
              groovy:
                expression: |-
                  def obj = [:]
                  obj.FirstName = "M"
                  obj.LastName = "K"

                  result = obj

                  return result
        - marshal:
            json:
              library: jackson
              prettyPrint: true
        - to:
            uri: https://webhook.sitessss/b7562774-4939-4e90-9337-5b9dd4d2ff1d

0 Comments

We have a project which is using the older DocumentClient based CosmosDB SDK. Getting this project to communicate with a Docker hosted CosmosDB emulator turned out to be a hassle.

The new SDK contains the following functionality for ignoring certificate issues:

CosmosClientOptions cosmosClientOptions = new CosmosClientOptions()
{
    HttpClientFactory = () =>
    {
        HttpMessageHandler httpMessageHandler = new HttpClientHandler()
        {
            ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator
        };

        return new HttpClient(httpMessageHandler);
    },
    ConnectionMode = ConnectionMode.Gateway
};

CosmosClient client = new CosmosClient(endpoint, authKey, cosmosClientOptions);

But the DocumentClient works little differently. But the good thing is that we can pass in a HttpClientHandler when creating the DocumentClient. So to ignore the cert issues when developing locally, one can use:

                    var handler = new HttpClientHandler();
                    handler.ClientCertificateOptions = ClientCertificateOption.Manual;
                    handler.ServerCertificateCustomValidationCallback = 
                        (httpRequestMessage, cert, cetChain, policyErrors) =>
                        {
                            return true;
                        };

                    client = new DocumentClient(
                        new Uri(endPointUrl),
                        primaryKey,
                        handler, connectionPolicy
                    );

If you need to configure serializer settings and the http client handler, things are a bit harder as there is not suitable public constructor in DocumentClient for configuring both. Reflection to rescue:

                    var handler = new HttpClientHandler();
                    handler.ClientCertificateOptions = ClientCertificateOption.Manual;
                    handler.ServerCertificateCustomValidationCallback = 
                        (httpRequestMessage, cert, cetChain, policyErrors) =>
                        {
                            return true;
                        };

                    client = new DocumentClient(
                        new Uri(endPointUrl),
                        primaryKey,
                        handler, connectionPolicy
                    );
                    
                    var prop = client.GetType().GetField("serializerSettings", System.Reflection.BindingFlags.NonPublic
                                                               | System.Reflection.BindingFlags.Instance);
                    prop.SetValue(client, serializerSettings);

logo_2Event Framework is an Open Source CloudEvents Framework for .NET applications. You include it in your .NET Core 3.1/.NET 6 app and it helps you to create, receive, send and to handle CloudEvents. After reaching 1.0.0-alpha.0.100 (with the first alpha dating back to early 2020), Event Framework is now available in beta form, with the 1.0.0-beta.1.1 release.

How to get started

The easiest way to get started with Event Framework is to include the package Weikio.EventFramework.AspNetCore in your ASP.NET Core based application and then to register the required bits and pieces into service container:

services.AddEventFramework()

Main features

The main features of the Event Framework include:

1. Create CloudEvents using CloudEventCreator
2. Send & Receive CloudEvents using Channels and Event Sources
3. Build Event Flows

Weikio Scenarios

Here’s a short example of each of those:

Creating CloudEvents

Event Framework includes CloudEventCreator which can be used to transform .NET objects into CloudEvents. It can be customized and used through either a static CloudEventCreator.Create-method or using a CloudEventCreator instance.

var obj = new CustomerCreated(Guid.NewGuid(), "Test", "Customer");

// Object to event
var cloudEvent = CloudEventCreator.Create(obj);

// Object to event customization
var cloudEventCustomName = CloudEventCreator.Create(obj, eventTypeName: "custom-name");

For more examples, please see the following tests in the Github repo: https://github.com/weikio/EventFramework/tree/master/tests/unit/Weikio.EventFramework.UnitTests

Sending CloudEvents

Event Framework uses Channels when transporting and transforming events from a source to an endpoint. Channels have adapters, components and endpoints (+ interceptors) which are used to process an event. Channels can be build using a fluent builder or more manually.

https://mermaid.ink/img/pako:eNp1UctOwzAQ_JXI57YSHHNAKhAEUgWI5ujLJt40RvE68gMEVf-dTd2UkghbsmbH45ldeS9qq1DkounsZ92CC9nmTdLquShXL9U71mG5vLlrgQg7SSfA1GNZvl7W6-_ocIvuQ9d4G70kH6udg77Nzo8zXko7ttSWjikDs1bQB3R-iLGmt4QUfLo6W4ya_zxSowk_FRQNOqg6TASSmthNc-aGG7vjfh50x6mJSZjJ0gH5xjozcf81ZVFBqrd6PsiEnwePgqu_5fVF2PEYtlgIHtSAVvx9-0EhRWjRoBQ5Q4UNxC5IIenA0tgrCFgoHawTeQOdx4WAGOz2i2qRBxdxFN1r4HbNSXX4AdpIuLI

Here's an example where a channel is created using the fluent builder with a single HTTP endpoint. Every object sent to this channel is transformed to CloudEvent and then delivered using HTTP:

var channel = await CloudEventsChannelBuilder.From("myHttpChannel")
    .Http("https://webhook.site/3bdf5c39-065b-48f8-8356-511b284de874")
    .Build(serviceProvider);

await channel.Send(new CustomerCreatedEvent() { Age = 50, Name = "Test User" });

In many situations in your applications you don’t send messages directly into the channel. Instead you inject ICloudEventPublisher into your controller/service and use it to publish events to a particular channel or to the default channel:

public IntegrationEndpointService(ICloudEventPublisher eventPublisher)
{
    _eventPublisher = eventPublisher;
}

...

await _eventPublisher.Publish(new EndpointCreated()
{
    Name = endpoint.Name,
    Route = endpoint.Route,
    ApiName = endpoint.ApiName,
    ApiVersion = endpoint.ApiVersion,
    EndpointId = result.Id.GetValueOrDefault(),
});

Receiving CloudEvents

Event Framework supports Event Sources. An event source can be used to receive events (for example: HTTP, Azure Service Bus) but an event source can also poll and watch changes happening in some other system (like local file system).

es_gif

Here's an example where HTTP and Azure Service Bus are used to receive events in ASP.NET Core and then logged:

services.AddEventFramework()
    .AddChannel(CloudEventsChannelBuilder.From("logChannel")
        .Logger())
    .AddHttpCloudEventSource("events")
    .AddAzureServiceBusCloudEventSource(
        "Endpoint=sb://sb.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=YDcvmuL4=",
        "myqueue");

services.Configure<DefaultChannelOptions>(options => options.DefaultChannelName = "logChannel");

Building Event Flows

Event Sources and Channels can be combined into Event Flows. Event Flows also support branching and subflows.

Here's an example where an event source is used to track file changes in a local file system and then all the created files are reported using HTTP:

var flow = EventFlowBuilder.From<FileSystemEventSource>(options =>
    {
        options.Configuration = new FileSystemEventSourceConfiguration() { Folder = @"c:\\temp\\myfiles", Filter = "*.bin" };
    })
    .Filter(ev => ev.Type == "FileCreatedEvent" ? Filter.Continue : Filter.Skip)
    .Http("https://webhook.site/3bdf5c39-065b-48f8-8356-511b284de874");

services.AddEventFramework()
    .AddEventFlow(flow);

Coming Next

Current document is lacking and samples also need work. The hope is to be able to include as many components and event sources as possible and for these, we’re looking at maybe using Apache Camel to bootstrap things.

Project Home

Please visit the project home site at https://weik.io/eventframework for more details. Though for now, the details are quite thin.

Source code

Source code for Event Framework is available from https://github.com/weikio/EventFramework.

af_createRecently I blogged about API Framework, an open source project which aims to make building ASP.NET Core backends more flexible. One of the core functionalities API Framework provides is the ability to add new endpoints to the backend runtime, without having to rebuild and restart the system. This blog introduces how this can be done.

The Goal

The goal is to have ASP.NET Core based app which can add new endpoints using an UI. When a new endpoint is added, we want the backend’s OpenAPI document to automatically update.

Source Code

The source code for this blog post is available from GitHub: https://github.com/weikio/ApiFramework.Samples/tree/main/misc/RunTime

The Starting Point

We use ASP.NET Core 3.1 Razor Page application as a starting point. The template has been modified so that we can use Blazor components inside the Razor Pages. Blazor documentation provides a good step-by-step tutorial for this.

1. Adding API Framework

API Framework (AF) is the tool which allows us to add and remove endpoints runtime. API Framework is available as a Nuget package and there is a specific package which works as a good starting point for ASP.NET Core Web API projects in 3.1 apps but as we want to include Razor Pages, Blazor components and controllers inside a single ASP.NET Core app, it’s better to use the Weikio.ApiFramework.AspNetCore package.

The package which we want to add is Weikio.ApiFramework.AspNetCore. The current version is 1.1.0 so let’s add that to the application in addition to NSwag:

  <ItemGroup>
    <PackageReference Include="Weikio.ApiFramework.AspNetCore" Version="1.1.0"/>
    <PackageReference Include="NSwag.AspNetCore" Version="13.8.2" />
  </ItemGroup>

Then Startup.ConfigureServices is changed to include controllers, AF and OpenAPI documentation (provided by NSwag):

        public void ConfigureServices(IServiceCollection services)
        {
            services.AddRazorPages();
            services.AddServerSideBlazor();
            
            services.AddControllers();
            services.AddApiFramework();
            services.AddOpenApiDocument();
        }

And finally Startup.Configure is configured to include Controllers and Blazor & OpenAPI. AF uses the default controller mappings:

        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            app.UseDeveloperExceptionPage();

            app.UseHttpsRedirection();
            app.UseStaticFiles();

            app.UseRouting();

            app.UseAuthorization();

            app.UseOpenApi();
            app.UseSwaggerUi3();

            app.UseEndpoints(endpoints =>
            {
                endpoints.MapControllers();
                endpoints.MapRazorPages();
                endpoints.MapBlazorHub();
            });
        }

Now when we run the app, we shouldn’t see any obvious changes:

image

But if we navigate to /swagger, we should see all the endpoints provided by our app, which is none at this moment:

image

Now that we have the AF running, it’s time to build API.

2. Building an API

API in API Framework is a reusable component with a name, a version and the specific functionality the API provides. APIs can be build using plain C# or using tools like delegates, Roslyn scripts and more. The APIs are often redistributed as Nuget packages.

Let’s build the simplest API: The classic Hello World. We can use C# to build the API and the API is just a POCO, there’s no need to add routes or HTTP methods:

    public class HelloWorldApi
    {
        public string SayHello()
        {
            return "Hello from AF!";
        }
    }

And that’s it. Now, in AF an API itself is not accessible using HTTP. Instead, we can create 1 or many endpointsfrom the API. These endpoints can then be accessed using HTTP. In this blog we want to create an UI which can be used to create these endpoints runtime.

If we now run the application, nothing has changed as we haven’t created any endpoints.

API Framework keeps a catalog of available APIs. This means that we have to either register the APIs manually (using code or configuration) or we can let AF to auto register the APIs based on conventions. In this tutorial we can let AF to auto register the APIs and this can be turned on using the options:

            services.AddApiFramework(options =>
            {
                options.AutoResolveApis = true;
            });

Now we are in a good position. We have an API and we have registered it into AF’s API Catalog (or rather, we have let AF register it automatically). As we still haven’t created any endpoints, running the app doesn’t provide any new functionality.

The next step is to build the Blazor component for creating endpoints runtime.

3. Creating the UI

We have the AF configured and the first API ready and waiting so it’s time to build the UI which can be used to add endpoints runtime.

We start by creating an UI which lists all the APIs in AF’s API catalog. For this we need a new Blazor-component which we call EndpointManagement.razor:

<h3>API Catalog</h3>

@code {
    
}

API Catalog is browsable using the IApiProvider-interface. Here’s a component which gets all the APIs and creates a HTML table of them:

@using Weikio.ApiFramework.Abstractions

<h3>API Catalog</h3>

<table class="table table-responsive table-striped">
    <thead>
    <tr>
        <th>Name</th>
        <th>Version</th>
    </tr>
    </thead>
    <tbody>
    @foreach (var api in ApiProvider.List())
    {
        <tr>
            <td>@api.Name</td>
            <td>@api.Version</td>
        </tr>
    }
    </tbody>
</table>

@code {

    [Inject]
    IApiProvider ApiProvider { get; set; }

    protected override void OnInitialized()
    {
        base.OnInitialized();
    }

}

And now we can add the component to Index.cshtml:

@page
@model IndexModel
@{
    ViewData["Title"] = "Home page";
}

<div class="text-center">
    <h1 class="display-4">Welcome</h1>
    <p>Learn about <a href="https://docs.microsoft.com/aspnet/core">building Web apps with ASP.NET Core</a>.</p>
</div>

<component type="typeof(EndpointManagement)" render-mode="ServerPrerendered" />

And finally we have something new visible to show. Run the app and… the API Catalog table is empty:

image

The reason for this is that the API Catalog is initialized in a background thread. AF supports dynamically created APIs and it sometimes can take minutes to create these so by default, AF does things on background.

F5 should fix the situation:

image

And there we have it, a catalog of APIs. If we at this point navigate to /swagger we can see that the list is still empty as we haven’t created any endpoints from the API:

image

Let’s fix that and add the required UI for listing and creating the endpoints. For that we need IEndpointManager:

    [Inject]
    IEndpointManager EndpointManager { get; set; }

UI should be simple: user can select the API by clicking the row and then she can enter the route to the endpoint & click Create.

For this we add couple new properties to the component:

    ApiDefinition SelectedApi { get; set; }
    string EndpointRoute { get; set; }

And then a really simple UI which can be used to create the endpoint:

<h3>Create Endpoint</h3>
<strong>Selected API: </strong> @SelectedApi <br/>
<strong>Route: </strong><input @bind="EndpointRoute"/><br/>
<div class="mt-3"></div>
<button class="btn btn-primary" @onclick="Create">Create</button>

Finally, all that is left is the implementation of the Create-method. Creating an endpoint in runtime takes couple of things:

  • First we need to define the endpoint. The endpoint always contains the route and the API but it also can contain configuration. This means that we could create a configurable API and then create multiple endpoints from that single API, each having a different configuration.
  • Then we need to provide the new endpoint definition to the IEndpointManager.
  • Finally we tell IEndpointManager to Update. This makes the new endpoint accessible. This way it is possible to first add multiple endpoints and then with a single Update call make them all visible at the same time.

Here’s how we can define the endpoint:

var endpoint = new EndpointDefinition(EndpointRoute, SelectedApi);

Then we can use CreateAndAdd method to actually the create the endpoint using the definition and to add it into the system with a single call:

        EndpointManager.CreateAndAdd(endpoint);

And now we just ask EndpointManager to update itself. At this point it goes through all the endpoints and initializes the new ones and updates the changed ones:

EndpointManager.Update();

Here’s the full Create-method:

    private void Create()
    {
        var endpoint = new EndpointDefinition(EndpointRoute, SelectedApi);
        
        EndpointManager.CreateAndAdd(endpoint);
        
        EndpointManager.Update();
    }

And here’s what we see when we run the app:

image

Now we can actually test if our setup works. Click the API from API Catalog, then provide route (for example /test) and finally click Create.

image

Maybe something happened, even though our UI doesn’t give any feedback. Navigating to /api/test should tell us more… And there it is!

image

Now if we check /swagger we should see the new endpoint:

image

Excellent. Try adding couple more endpoints, like /hello and /world and make sure things work as expected:

image

That’s it. We now have an ASP.NET Core backend which can be modified runtime.

If you wonder why the endpoint is available form /api/test and not from /test, it is because the AF by default uses the /api prefix for all the endpoints. This can be configured using options.

Conclusion

In this blog we modified an ASP.NET Core 3.1 based application so that it supports runtime changes. Through these runtime changes we can add new endpoints without having to rebuild or to restart the app. The code is available from https://github.com/weikio/ApiFramework.Samples/tree/main/misc/RunTime.

This blog works more as an introduction to the subject as we used only one API and that was the simplest possible, the Hello World. But we will expand from this and here’s some ideas of things to explore, all supported by API Framework:

  • Removing an endpoint runtime
  • Creating endpoints from an API which requires configuration
  • Listing endpoints
  • Viewing endpoint’s status
  • Creating the APIs (not endpoints) runtime using C#
  • Creating an endpoint runtime from an API distributed as a Nuget package.

To read more about API Framework, please visit the project’s home pages at https://weik.io/apiframework. The AF’s source code is available from GitHub and documentation can be found from https://docs.weik.io/apiframework/.

LogoAPI Framework is a new open source project (Apache License 2.0) for ASP.NET Core with the aim of bringing more flexibility and more options to building ASP.NET Core based OpenAPI backends. With API Framework your code and your resourcesbecome the OpenAPI endpoints.

With API Framework the endpoints can be build using POCOs, Roslyn Script and Delegates and the reusable APIs can be distributed as plugins using Nuget. The APIs can be added runtime, without having to restart or to rebuild the application. The flexible plugin system with a built-in support for plugins means that API Framework can be used as the secure OpenAPI gateway into other systems. With plugins you can generate OpenAPI endpoints from databases, file systems, Azure blob storage, and more.

API Framework version 1.0.0 is now available from Nuget: https://www.nuget.org/packages/weikio.apiframework.aspnetcore. The project’s home pages is available from https://weik.io/apiframework.

Main Features

“Everything is an OpenAPI” mentality means that there’s multiple ways of building your backend’s endpoints. Plain C# and delegates are supported but so are your database and your browser and many more. Your code and your resources are the OpenAPI endpoints.

You can also integrate API Framework with your existing Controller & Action based app. API Framework is built on top of the MVC-functionality meaning it can take advantage of many of the features provided by ASP.NET Core.

Here’s a short summary of the major features of API Framework:

  • Everything is an OpenAPI:The API can be anything: Database, local files. Even the web browser. APIs can be created with C#, Roslyn Scripts and using delegates. These endpoint provide the full OpenAPI specs, meaning if you have a table called Inventory in your database, your OpenAPI specification will have the same Inventory available for use.
  • Runtime Changes: APIs and endpoints can be configured runtime, when the application is running. No need to restart the system to add a new endpoint.
  • Plugin Support:API Frameworks supports plugins. Nuget.org contains multiple ready made plugins, like SQL Server and Local Files. Custom plugins can be created and shared using Nuget.

There’s many more features, including support for health checks and async JSON streams. More information is available from the project’s home pages.

Terminology & Main Concepts

With API Framework there’s two main concepts:

API

In API Framework each API has two properties: Name and Version. And the actual functionality they provide.

Most often API is a C# class or a .NET type. But you can also use assemblies, Nuget packages, Roslyn scripts and delegates as APIs. Often the aim is to make APIs reusable and they can be shared using Nuget.

Endpoint

An API itself is not accessible through a web request. That's the work of an Endpoint. Endpoint gives your API a route (for example '/helloworld') and a configuration (if needed).

You can create multiple endpoints from each API and every endpoint can have different configuration.

Quick Start

Here’s a quick start, adapted from the documentation at https://docs.weik.io/apiframework/quickstart.html. The starting point is a blank ASP.NET Core 3.1 web api app.

This quick start shows how to build integration/data hub style backend using API Framework’s plugins. When the quick start is ready, we have an ASP.NET Core based backend which provides access to SQL Server and to an another OpenAPI backend (the PetShop).

In this quick start the APIs and Endpoints are added through code. Configuration through appsettings.json (IConfiguration) and through runtime REST endpoints are also supported.

1. Add the required package for API Framework:

        <PackageReference Include="Weikio.ApiFramework.AspNetCore.StarterKit" Version="1.0.0"/>

2. Add the SQL Server and OpenAPI plugins:

        <PackageReference Include="Weikio.ApiFramework.Plugins.SqlServer" Version="2.0.0" />
        <PackageReference Include="Weikio.ApiFramework.Plugins.OpenApi" Version="1.0.0" />

3. Modify Startup.Configure by adding API Framework & required APIs and Endpoints:

        public void ConfigureServices(IServiceCollection services)
        {
            services.AddControllers();

            services.AddApiFrameworkStarterKit()
                .AddOpenApi("/pets", new ApiOptions()
                {
                    SpecificationUrl = "https://petstore3.swagger.io/api/v3/openapi.json",
                    ApiUrl = "https://petstore3.swagger.io/api/v3"
                })
                .AddSqlServer("/eshop",
                    new SqlServerOptions()
                    {
                        // This provides a read only access to the db used in the documentations
                        ConnectionString =
                            "Server=tcp:adafydevtestdb001.database.windows.net,1433;User ID=docs;Password=3h1@*6PXrldU4F95;Integrated Security=false;Initial Catalog=adafyweikiodevtestdb001;"
                    });
        }

4. Run the application, wait a few second (the endpoints are generated on the background) and navigate tohttps://localhost:5001/swagger

image

The SQL Server database can be queried and filtered and the OpenAPI specs contains the schema for the database:

image

Getting Started

The quick start above show one approach for using API Framework. Best way to learn more about API Framework is through the following resources:

Use Cases

As API Framework adds more flexibility into building and running ASP.NET Core backends, there’s different use cases where it can be helpful:

  • 24/7 Backends:API Framework can be used to create ASP.NET Core backends which can be updated runtime. APIs and endpoints can be added and updated when the system is running. As an example, it’s possible to install API through Nuget and then configure one or more endpoints based on that API. All this runtime.
  • Integration Layers:API Framework can be used as the secure OpenAPI gateway into other systems. With plugins you can generate OpenAPI endpoints from databases, file systems, Azure blob storage, and more. This is helpful in scenarios where you want build integration/data hub style application.
  • More options for building APIs:As mentioned, with API Framework it is possible to build APIs using Roslyn, Delegates and more. This means it is entirely possible to create a backend using a folder of Roslyn Script files.

Current Status

Today is the day of the 1.0.0 release. The work continues, especially with the samples and with documentation. Currently there’s about 12 samples available, showcasing many of the features but there’s more to add. The documentation is little farther behind.

Some of the plugins released today are more advanced than the others. For example the database plugins are quite advanced but something like the Files-plugin is currently very barebones.

Project Home

API Framework can be found from Github and from https://weik.io/ApiFramework.