Skip to main content
Version: 3.1 (latest)

Publishing Data Assets for Multi-Stage Computation

Federated Compute enables algorithms to run across distributed datasets simultaneously and aggregates the results into a single final output

For a Federated Compute scenario to work, you'll need to publish at least 4 different data assets:

  • An Algorithm for Aggregation: the algorithm on the second stage of computation, responsible for combining the results into a unified output.
  • An Algorithm for Federation: the algorithm on the first stage of the computation, responsible for running across distributed datasets.
  • The Dataset(s) (Compute type): the assets in which the Algorithm for Federation will run on.
  • Any placeholder dataset to define the location of computation (Compute type).

2052

Step 1: Publish the Algorithm for Aggregation​

  1. Log in to the marketplace.
  2. From the left navigation bar, go to Create > Algorithm.
  3. Under Algorithm Type, select 'Aggregate'.
  4. Key in all other required information for publishing an algorithm. For more details, please refer to Publishing an Algorithm.

Step 2: Publish the Algorithm for Federation​

  1. Log in to the marketplace.
  2. From the left navigation bar, go to Create > Algorithm.
  3. Under Algorithm Type, select 'Federated'.
  4. Under Define your aggregate algorithm, select the Aggregate Algorithm that you want to use or just published from Step 1.
  5. Key in all other required information for publishing an algorithm. For more details, please refer to Publishing an Algorithm.

Step 3: Publish a Dataset (Compute)​

  1. Log in to the marketplace.
  2. From the left navigation bar, go to Create > Dataset.
  3. Under the Access Type, click on 'Compute'.
  4. Do NOT enable Aggregate only. Keep it as 'Disable'.
  5. Under Compatible Algorithm(s), select the Federated Algorithm that you want to use or just published from Step 2.
  6. Key in all other required information for publishing a compute-type dataset. For more details, please refer to Publishing a Dataset for Compute-to-Data.
  7. Repeat the same steps to publish additional datasets for your usage.

Step 4: Publish a Placeholder Dataset​

  1. Log in to the marketplace.
  2. From the left navigation bar, go to Create > Dataset.
  3. Under the Access Type, click on 'Compute'.
  4. Enable Aggregate only option.
  5. Make sure to select the Edge Node you wish to use to run the computation during the aggregation stage.
  6. Under Compatible Algorithm(s), select the Aggregate Algorithm that you want to use or just published from Step 1.
  7. Key in all other required information for publishing a compute-type dataset. For more details, please refer to Publishing a Dataset for Compute-to-Data.

Having this 'Placeholder' dataset helps the Aggregated algorithm to know where to run the computation during the aggregation step of the multi-stage computing.