streamnative.io
Help Center
Login
  1. StreamNative Private Cloud
  2. Configure Private Cloud
  3. Security
  4. Authentication

Multiple private keys support for JWT token authentication

StreamNative Pulsar extends the JWT token authentication to support multiple private keys AuthenticationProvider AuthenticationProviderMultipleSignKeyToken, and implement automatic reloading of configuration files into the cache without restarting the broker.

The multiple multiple private keys will be maintained in a JSON file and issue new JWT token needs to include a kid field in the header to match the private key for signature verification.

Before you begin

Create multiple secret keys

  • Create the default secret key

    pulsarctl token create-secret-key -a HS256 --output-file default-secret.key

cat default-secret.key | base64
GRgrvdFdNijm0xPompgfJGwE77ifM+U31exWERDEiB0=

  • Create the second secret key

    pulsarctl token create-secret-key -a HS256 --output-file kid-secret.key

cat kid-secret.key| base64
IeNNHJmTM6icg71/D6aFQ/buslNV+FZWT/TiJRDZpYo=

  • Create a JSON to include the multiple secret keys, key is kid, value is the corresponding secret key encoded by base64

    {
  "KID1": "Cysl6yGZb6xFygc0hB5Pt+ha1ZXpt7jmFU3CR1AS2os=",
  "DEFAULT_KID": "nVXSRdlQm4wbgxiS4/MHjxgLqyL9iF2oEXdq9E6iIeo="
}

  • Create the K8s Secret for the multi secret key JSON

    kubectl create secret generic multi-secret-key --from-file=secret-keys.json

Create broker and proxy tokens

  • Use the default secret key to generate broker admin token

    pulsarctl token create -a HS256 --secret-key-file default-secret.key --subject broker-admin
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJicm9rZXItYWRtaW4ifQ.bi1Bx9Wg-4HOA-xvAPJQQbf-J70u359TL_8_GqSdC6E

  • Create the K8s Secret for the tokens

    kubectl create secret generic broker-admin --from-literal=broker-admin=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJicm9rZXItYWRtaW4ifQ.bi1Bx9Wg-4HOA-xvAPJQQbf-J70u359TL_8_GqSdC6E

    kubectl create secret generic proxy-admin --from-literal=proxy-admin=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJwcm94eS1hZG1pbiJ9.ELqIKJlb0ISShCTBmhV0MenPbBLG0xkRk1mrfbnWT8s

Configure PulsarBroker and PulsarProxy

PulsarBroker

spec:
  image: streamnative/private-cloud:3.3.0.7
  replicas: 1
  zkServers: private-cloud-zk:2181
  config:
    clusterName: private-cloud
    custom:
      managedLedgerDefaultAckQuorum: '1'
      managedLedgerDefaultEnsembleSize: '1'
      managedLedgerDefaultWriteQuorum: '1'
      authenticationEnabled: 'true'
      authenticateOriginalAuthData: 'true'
      authenticationProviders: 'io.streamnative.pulsar.broker.authentication.AuthenticationProviderMultipleSignKeyToken'
      brokerClientAuthenticationPlugin: 'org.apache.pulsar.client.impl.auth.AuthenticationToken'
      superUserRoles: 'admin,proxy-admin,broker-admin,admin-approle'
      proxyRoles: 'proxy-admin'
      authorizationEnabled: 'true'
      authorizationProvider: 'org.apache.pulsar.broker.authorization.PulsarAuthorizationProvider'
  pod:
    resources:
      requests:
        cpu: 200m
        memory: 512Mi
    securityContext:
      runAsNonRoot: true
    secretRefs:
      - mountPath: /mnt/secrets
        secretName: multi-secret-key
      - mountPath: /mnt/tokens
        secretName: broker-admin
    vars:
      - name: brokerClientAuthenticationParameters
        value: 'file:///mnt/tokens/broker-admin'
      - name: tokenSecretKey
        value: 'file:///mnt/secrets/secret-keys.json'
  • [1] spec.config.cusom: authenticationProviders should use the io.streamnative.pulsar.broker.authentication.AuthenticationProviderMultipleSignKeyToken

PulsarProxy

spec:
  image: streamnative/private-cloud:3.3.0.7
  replicas: 1
  config:
    custom:
      authenticationEnabled: 'true'
      authenticateOriginalAuthData: 'true'
      forwardAuthorizationCredentials: 'true'
      authenticationProviders: 'io.streamnative.pulsar.broker.authentication.AuthenticationProviderMultipleSignKeyToken'
      brokerClientAuthenticationPlugin: 'org.apache.pulsar.client.impl.auth.AuthenticationToken'
      superUserRoles: 'proxy-admin'
  brokerAddress: private-cloud-broker
  pod:
    annotations:
      prometheus.io/port: '8080'
      prometheus.io/scrape: 'true'
    resources:
      requests:
        cpu: 200m
        memory: 512Mi
    securityContext:
      runAsNonRoot: true
    secretRefs:
      - mountPath: /mnt/secrets
        secretName: multi-secret-key
      - mountPath: /mnt/tokens
        secretName: proxy-admin
    vars:
      - name: brokerClientAuthenticationParameters
        value: 'file:///mnt/tokens/proxy-admin'
      - name: tokenSecretKey
        value: 'file:///mnt/secrets/secret-keys.json'

Create client tokens

  • Create a token by the default secret key

    pulsarctl token create -a HS256 --secret-key-file default-secret.key --subject test-client
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ0ZXN0LWNsaWVudCJ9.BC8hq47exYyCirBCP5AlIDzFQtnxhCPrv94BHeQ08to

  • Create a token by the kid secret key

    Note

    The kid=KID1 in the headers should be consistent with the secret-keys.json defination.

    pulsarctl token create -a HS256 --secret-key-file kid-secret.key --subject test-client --headers kid=KID1
eyJhbGciOiJIUzI1NiIsImtpZCI6ImtpZDEiLCJ0eXAiOiJKV1QifQ.eyJzdWIiOiJ0ZXN0LWNsaWVudCJ9.y5qXOgb_ibTQD09nVaS6sr4bGI8caByY_FnJXuhq4_4

  • Grant permission to the role

    pulsarctl --token "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJicm9rZXItYWRtaW4ifQ.bi1Bx9Wg-4HOA-xvAPJQQbf-J70u359TL_8_GqSdC6E" \
namespaces grant-permission --role test-client --actions produce --actions consume public/default

  • Use the token generated by the default secret key to produce data

    bin/pulsar-client --auth-params token:eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ0ZXN0LWNsaWVudCJ9.BC8hq47exYyCirBCP5AlIDzFQtnxhCPrv94BHeQ08to --auth-plugin org.apache.pulsar.client.impl.auth.AuthenticationToken produce default-secret-token -m "test-with-multisecretKey" -n 15

2024-05-29T21:46:20,293+0800 [pulsar-client-io-1-5] INFO  org.apache.pulsar.client.impl.ClientCnx - [id: 0xc1116ccc, L:/198.18.0.1:62077 - R:/172.171.186.162:6650] Connected through proxy to target broker at private-cloud-broker-0.private-cloud-broker-headless.default.svc.cluster.local:6650
2024-05-29T21:46:20,854+0800 [pulsar-client-io-1-5] INFO  org.apache.pulsar.client.impl.ProducerImpl - [default-secret-token] [null] Creating producer on cnx [id: 0xc1116ccc, L:/198.18.0.1:62077 - R:/172.171.186.162:6650]
2024-05-29T21:46:21,263+0800 [pulsar-client-io-1-5] INFO  org.apache.pulsar.client.impl.ProducerImpl - [default-secret-token] [private-cloud-0-3] Created producer on cnx [id: 0xc1116ccc, L:/198.18.0.1:62077 - R:/172.171.186.162:6650]
2024-05-29T21:46:25,671+0800 [main] INFO  org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - [default-secret-token] [private-cloud-0-3] --- Publish throughput: 2.66 msg/s --- 0.00 Mbit/s --- Latency: med: 267.000 ms - 95pct: 441.000 ms - 99pct: 441.000 ms - 99.9pct: 441.000 ms - max: 441.000 ms --- BatchSize: med: 1.000 - 95pct: 1.000 - 99pct: 1.000 - 99.9pct: 1.000 - max: 1.000 --- MsgSize: med: 24.000 bytes - 95pct: 24.000 bytes - 99pct: 24.000 bytes - 99.9pct: 24.000 bytes - max: 24.000 bytes --- Ack received rate: 2.66 ack/s --- Failed messages: 0 --- Pending messages: 0
2024-05-29T21:46:25,928+0800 [pulsar-client-io-1-5] INFO  org.apache.pulsar.client.impl.ProducerImpl - [default-secret-token] [private-cloud-0-3] Closed Producer
2024-05-29T21:46:25,930+0800 [main] INFO  org.apache.pulsar.client.impl.PulsarClientImpl - Client closing. URL: pulsar://172.171.186.162:6650/
2024-05-29T21:46:25,935+0800 [pulsar-client-io-1-5] INFO  org.apache.pulsar.client.impl.ClientCnx - [id: 0xc1116ccc, L:/198.18.0.1:62077 ! R:/172.171.186.162:6650] Disconnected
2024-05-29T21:46:25,935+0800 [pulsar-client-io-1-3] INFO  org.apache.pulsar.client.impl.ClientCnx - [id: 0xe6914d5b, L:/198.18.0.1:62075 ! R:/172.171.186.162:6650] Disconnected
2024-05-29T21:46:27,989+0800 [main] INFO  org.apache.pulsar.client.cli.PulsarClientTool - 15 messages successfully produced

  • Use the token generated by the KID1 secret key to produce data

    bin/pulsar-client --auth-params token:eyJhbGciOiJIUzI1NiIsImtpZCI6IktJRDEiLCJ0eXAiOiJKV1QifQ.eyJzdWIiOiJ0ZXN0LWNsaWVudCJ9.YbsyE54ulm9ioMx2xZBqLgCNrYmjk2o6wxR5WOwaPSE --auth-plugin org.apache.pulsar.client.impl.auth.AuthenticationToken produce test -m "test-with-multisecretKey" -n 15

2024-05-29T21:46:20,293+0800 [pulsar-client-io-1-5] INFO  org.apache.pulsar.client.impl.ClientCnx - [id: 0xc1116ccc, L:/198.18.0.1:62077 - R:/172.171.186.162:6650] Connected through proxy to target broker at private-cloud-broker-0.private-cloud-broker-headless.default.svc.cluster.local:6650
2024-05-29T21:46:20,854+0800 [pulsar-client-io-1-5] INFO  org.apache.pulsar.client.impl.ProducerImpl - [default-secret-token] [null] Creating producer on cnx [id: 0xc1116ccc, L:/198.18.0.1:62077 - R:/172.171.186.162:6650]
2024-05-29T21:46:21,263+0800 [pulsar-client-io-1-5] INFO  org.apache.pulsar.client.impl.ProducerImpl - [default-secret-token] [private-cloud-0-3] Created producer on cnx [id: 0xc1116ccc, L:/198.18.0.1:62077 - R:/172.171.186.162:6650]
2024-05-29T21:46:25,671+0800 [main] INFO  org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - [default-secret-token] [private-cloud-0-3] --- Publish throughput: 2.66 msg/s --- 0.00 Mbit/s --- Latency: med: 267.000 ms - 95pct: 441.000 ms - 99pct: 441.000 ms - 99.9pct: 441.000 ms - max: 441.000 ms --- BatchSize: med: 1.000 - 95pct: 1.000 - 99pct: 1.000 - 99.9pct: 1.000 - max: 1.000 --- MsgSize: med: 24.000 bytes - 95pct: 24.000 bytes - 99pct: 24.000 bytes - 99.9pct: 24.000 bytes - max: 24.000 bytes --- Ack received rate: 2.66 ack/s --- Failed messages: 0 --- Pending messages: 0
2024-05-29T21:46:25,928+0800 [pulsar-client-io-1-5] INFO  org.apache.pulsar.client.impl.ProducerImpl - [default-secret-token] [private-cloud-0-3] Closed Producer
2024-05-29T21:46:25,930+0800 [main] INFO  org.apache.pulsar.client.impl.PulsarClientImpl - Client closing. URL: pulsar://172.171.186.162:6650/
2024-05-29T21:46:25,935+0800 [pulsar-client-io-1-5] INFO  org.apache.pulsar.client.impl.ClientCnx - [id: 0xc1116ccc, L:/198.18.0.1:62077 ! R:/172.171.186.162:6650] Disconnected
2024-05-29T21:46:25,935+0800 [pulsar-client-io-1-3] INFO  org.apache.pulsar.client.impl.ClientCnx - [id: 0xe6914d5b, L:/198.18.0.1:62075 ! R:/172.171.186.162:6650] Disconnected
2024-05-29T21:46:27,989+0800 [main] INFO  org.apache.pulsar.client.cli.PulsarClientTool - 15 messages successfully produced
Previous
JWT authentication
Next
Audit Log