Troubleshooting a service network

Typically, you can create a service network without referencing this troubleshooting guide. However, this guide provides some tips for situations when the service network does not perform as expected.

See Resolving common problems if you have encountered a specific issue using the skupper CLI.

A typical troubleshooting workflow is to check all the sites and create debug tarballs.

Checking sites

Using the skupper command-line interface (CLI) provides a simple method to get started with troubleshooting Skupper.

Procedure

  1. Check the site status:

    $ skupper status --namespace west

Skupper is enabled for namespace "west" in interior mode. It is connected to 2 other sites. It has 1 exposed services.

    The output shows:

    • A site exists in the specified namespace.

    • A link exists to two other sites.

    • A service is exposed on the service network and is accessible from this namespace.

  2. Check the link status:

    $ skupper link status --namespace east

Link link1 is active

    A link exists from the specified site to another site, meaning a token from another site was applied to the specified site.

    Running skupper link status on a connected site produces output only if a token was used to create a link.

  3. Check the service network:

    $ skupper network status --namespace west

Sites:
├─ [local] 05f8c38 - west
│  URL: 10.110.15.54
│  mode: interior
│  name: west
│  namespace: west
│  version: 1.0.2
│  ╰─ Services:
│     ╰─ name: backend
│        address: backend: 8080
│        protocol: tcp
╰─ [remote] 1537b82 - east
   URL: 10.97.26.100
   name: east
   namespace: east
   sites linked to: 05f8c38-west
   version: 1.0.2
   ╰─ Services:
      ╰─ name: backend
         address: backend: 8080
         protocol: tcp
         ╰─ Targets:
            ├─ name: backend-77f8f45fc8-smckp
            ├─ name: backend-77f8f45fc8-gh6tp
            ╰─ name: backend-77f8f45fc8-m58tg

    The output shows:

    • There are 2 sites on the service network, east and west.

    • Details for each site, for example the namespace names.

    • The original services that are exposed (Targets), in this case the three backend services exposed using the tcp protocol.

    • The services available on the service network, including the port number. For example, backend:8080.

Creating a skupper debug tarball

The debug tarball contains all the logs from the Skupper components for a site and provides detailed information to help debug issues.

  1. Create the debug tarball:

    $  skupper debug dump my-site

Skupper dump details written to compressed archive:  `my-site.tar.gz`

  2. You can expand the file using the following command:

    $ tar -xvf kind-site.tar.gz

k8s-versions.txt
skupper-versions.txt
skupper-router-deployment.yaml
skupper-router-867f5ddcd8-plrcg-skstat-g.txt
skupper-router-867f5ddcd8-plrcg-skstat-c.txt
skupper-router-867f5ddcd8-plrcg-skstat-l.txt
skupper-router-867f5ddcd8-plrcg-skstat-n.txt
skupper-router-867f5ddcd8-plrcg-skstat-e.txt
skupper-router-867f5ddcd8-plrcg-skstat-a.txt
skupper-router-867f5ddcd8-plrcg-skstat-m.txt
skupper-router-867f5ddcd8-plrcg-skstat-p.txt
skupper-router-867f5ddcd8-plrcg-router-logs.txt
skupper-router-867f5ddcd8-plrcg-config-sync-logs.txt
skupper-service-controller-deployment.yaml
skupper-service-controller-7485756984-gvrf6-events.txt
skupper-service-controller-7485756984-gvrf6-service-controller-logs.txt
skupper-site-configmap.yaml
skupper-services-configmap.yaml
skupper-internal-configmap.yaml
skupper-sasl-config-configmap.yaml

    These files can be used to provide support for Skupper, however some items you can check:

    versions

    See *versions.txt for the versions of various components.

    ingress

    See skupper-site-configmap.yaml to determine the ingress type for the site.

    linking and services

    See the skupper-service-controller-*-events.txt file to view details of token usage and service exposure.

Resolving common problems

The following issues and workarounds might help you debug simple scenarios when evaluating Skupper.

Cannot initialize skupper

If the skupper init command fails, consider the following options:

  • Check the load balancer.

    If you are evaluating Skupper on minikube, use the following command to create a load balancer:

    $ minikube tunnel

    For other Kubernetes flavors, see the documentation from your provider.

  • Initialize without ingress.

    This option prevents other sites from linking to this site, but linking outwards is supported. Once a link is established, traffic can flow in either direction. Enter the following command:

    $ skupper init --ingress none

    See Skupper init for more options.

Cannot link sites

To link two sites, one site must be accessible from the other site. For example, if one site is behind a firewall and the other site is on an AWS cluster, you must:

  1. Create a token on the AWS cluster site.

  2. Create the link on the site inside the firewall.

Cannot access Skupper console

Use skupper status to find the console URL.

See Using the Skupper console for more information on Accessing the console, including how to retrieve the username and password.

Cannot create a token for linking clusters

You might see the following note after using the skupper token create command:

Token written to <path> (Note: token will only be valid for local cluster)

This output indicates that the site was deployed without an ingress option. For example skupper init --ingress none. You must specify an ingress to allow sites on other clusters to link to your site.

You can also use the skupper token create command to check if an ingress was specified when the site was created.