Skip to content

Conversation

@KrishnaIyer
Copy link
Member

@KrishnaIyer KrishnaIyer commented Nov 1, 2023

Summary

Support migration from the Firefly LNS. Replaces #49.

Changes

Testing

With a The Things Uno.

  1. Curl the device using the API (to check the last frame counter)
curl "https://xxxxxxx/api/v1/devices/eui/0004A30B001BB199?auth=$FIREFLY_API_KEY"
{"device":{"updated_at":"2023-10-31T18:30:53.110455","tags":[],"skip_fcnt_check":false,"rx2_data_rate":0,"region":"EU868","override_location":false,"otaa":true,"organization_id":226,"organization":{"name":"alec","id":226,"description":null},"network_session_key":"1BA000AB1BAC28B83F2A091F7EA9ABED","name":"Migration Test OTAA","multicast_groups":[],"location":null,"inserted_at":"2023-10-31T18:11:08.544913","id":62268,"frame_counter":0,"eui":"0004A30B001BB199","device_class_id":null,"description":null,"deduplicate":false,"class_c":false,"application_session_key":"927A87F8B710479D594440227BBBDD64","application_key":"5105D918B6EEEF8A22AA92B39D9EC718","adr_limit":null,"address":"29559735"}}

This can also be checked via the UI
otaa-firefly

  1. Export
./main firefly device 0004A30B001BB199 --invalidate-keys

{"ids":{"device_id":"eui-0004a30b001bb199","application_ids":{"application_id":"my-test-app"},"dev_eui":"0004A30B001BB199","join_eui":"1111111111111111"},"name":"Migration Test OTAA","lorawan_version":"MAC_V1_0_2","lorawan_phy_version":"PHY_V1_0_2_REV_B","frequency_plan_id":"EU_863_870","supports_join":true,"root_keys":{"app_key":{"key":"5105D918B6EEEF8A22AA92B39D9EC718"}},"mac_settings":{"rx2_data_rate_index":0,"status_time_periodicity":"0s","status_count_periodicity":0,"desired_adr_ack_limit_exponent":"ADR_ACK_LIMIT_1"},"session":{"dev_addr":"29559735","keys":{"session_key_id":"bBRMrEnueD7zAxHwX4a2RA==","f_nwk_s_int_key":{"key":"1BA000AB1BAC28B83F2A091F7EA9ABED"},"app_s_key":{"key":"927A87F8B710479D594440227BBBDD64"}},"last_f_cnt_up":10,"started_at":"2023-10-31T18:41:35.080851Z"}}
  1. Import and check uplink (active session)
otaa-frame12
  1. Rejoin on TTS
otaa-rejoin-tts
Regressions

NA. This is new functionality.

Notes for Reviewers

  1. I've only tested one device for OTAA and ABP. Before merging, I will test a batch.
  2. I'll add docs after we agree on the code.
  3. I had one failed frame during the OTAA migration. Is this a frequency plan configuration issue? should I have used EU_863_870_TTN instead? I'm using a The Things Uno for testing.
DEBUG Failed to determine channel index of uplink {"ack": false, "adr": false, "adr_ack_req": false, "band_id": "EU_863_870", "bandwidth": 125000, "class_b": false, "data_rate": "lora:{bandwidth:125000  spreading_factor:7  coding_rate:\"4/5\"}", "dev_addr": "29559735", "error": "error:pkg/networkserver:uplink_channel_not_found (uplink channel not found)", "f_cnt_reset": false, "f_opts_len": 0, "f_port": 1, "frequency": 867700000, "full_f_cnt_up": 11, "grpc.method": "HandleUplink", "grpc.service": "ttn.lorawan.v3.GsNs", "last_f_cnt_up": 10, "m_type": "UNCONFIRMED_UP", "mac_count": 0, "mac_version": "MAC_V1_0_2", "major": "LORAWAN_R1", "namespace": "networkserver", "pending_session": false, "phy_payload_len": 14, "received_at": 1698777742.7678819, "request_id": "01HE3GPFDFN4CBZX9Q697JE83A", "spreading_factor": 7, "uplink_f_cnt": 11}

Does anything stand out to you here @adriansmares?

Checklist

  • Scope: The referenced issue is addressed, there are no unrelated changes.
  • Documentation: Relevant documentation is added or updated.
  • Changelog: Significant features, behavior changes, deprecations and fixes are added to CHANGELOG.md.

@KrishnaIyer KrishnaIyer added this to the Nov 2023 milestone Nov 1, 2023
@KrishnaIyer KrishnaIyer self-assigned this Nov 1, 2023
@KrishnaIyer KrishnaIyer marked this pull request as ready for review November 1, 2023 10:15
Copy link
Contributor

@adriansmares adriansmares left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DEBUG Failed to determine channel index of uplink {"ack": false, "adr": false, "adr_ack_req": false, "band_id": "EU_863_870", "bandwidth": 125000, "class_b": false, "data_rate": "lora:{bandwidth:125000 spreading_factor:7 coding_rate:"4/5"}", "dev_addr": "29559735", "error": "error:pkg/networkserver:uplink_channel_not_found (uplink channel not found)", "f_cnt_reset": false, "f_opts_len": 0, "f_port": 1, "frequency": 867700000, "full_f_cnt_up": 11, "grpc.method": "HandleUplink", "grpc.service": "ttn.lorawan.v3.GsNs", "last_f_cnt_up": 10, "m_type": "UNCONFIRMED_UP", "mac_count": 0, "mac_version": "MAC_V1_0_2", "major": "LORAWAN_R1", "namespace": "networkserver", "pending_session": false, "phy_payload_len": 14, "received_at": 1698777742.7678819, "request_id": "01HE3GPFDFN4CBZX9Q697JE83A", "spreading_factor": 7, "uplink_f_cnt": 11}

This is normal to a certain extent - you don't pass any MAC state while importing the end device, which means that the LNS builds a MAC state which has the current parameters with only the 'boot' channels - 867700000 is not a 'boot' channel.

The ttnv2 source can show you how to build the MAC state and how to edit it. In this case you can do the same current = desired trick and then work back the different parameters in Firefly.

Copy link
Member

@happyRip happyRip left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! I like how you've streamlined the config part.

Copy link
Member

@johanstokking johanstokking left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This JoinEUI thing is probably quite problematic, no? This only works if there's one JoinEUI per batch? And you either migrate one device, or you migrate all devices accessible with the API key?

I presume Firefly just takes the JoinEUI from the join-request, and uses the DevEUI as primary identifier?

So if there's a mix of JoinEUIs in a larger deployment, devices need to be migrated one by one?

@KrishnaIyer KrishnaIyer mentioned this pull request Nov 6, 2023
3 tasks
Comment on lines -42 to -61
logLevel := log.InfoLevel
if verbose, _ := cmd.Flags().GetBool("verbose"); verbose {
logLevel = log.DebugLevel
}
logHandler, err := log.NewZap("console")
if err != nil {
return err
}
logger = log.NewLogger(
logHandler,
log.WithLevel(logLevel),
)
rpclog.ReplaceGrpcLogger(logger)
ctx = log.NewContext(ctx, logger)

exportCfg.DevIDPrefix, _ = cmd.Flags().GetString("dev-id-prefix")
exportCfg.EUIForID, _ = cmd.Flags().GetBool("set-eui-as-id")
ctx = export.NewContext(ctx, exportCfg)

cmd.SetContext(ctx)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The context was previously never propagated to the subcommands and hence this logger and the exportconfig were never used.

I've fixed the export config but we don't need this logger since we use Zap in the source.

@KrishnaIyer
Copy link
Member Author

Retested the changes

  1. OTAA
    The device sends frame 3 on Firefly, is migrated to TTS and TTS receives frame 4 onwards correctly. Rejoin works also
tts-frame-4 tts-rejoin-after-migration ff-frame-3
  1. ABP
    The device sends frame 3 on Firefly, is migrated to TTS and TTS receives frame 4 onwards correctly
ff-frame-3 tts-frame-4

Copy link
Member

@happyRip happyRip left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great

@KrishnaIyer
Copy link
Member Author

This JoinEUI thing is probably quite problematic, no? This only works if there's one JoinEUI per batch? And you either migrate one device, or you migrate all devices accessible with the API key?

Yes indeed. The way I have documented this is to migrate devices in batches by creating a file with the Device EUIs and piping that into the devices command.
This way, all devices in the batch will have a common JoinEUI on TTS.

@KrishnaIyer KrishnaIyer merged commit d9c9b39 into master Nov 13, 2023
@KrishnaIyer KrishnaIyer deleted the feature/43-firefly branch November 13, 2023 11:12
@adriansmares adriansmares mentioned this pull request Mar 11, 2024
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants