Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V10 - Changes to Sentry Mode #260

Closed
bladec59 opened this issue Sep 15, 2019 · 176 comments
Closed

V10 - Changes to Sentry Mode #260

bladec59 opened this issue Sep 15, 2019 · 176 comments

Comments

@bladec59
Copy link

From early view of release notes it looks like Tesla is changing/adding a folder to store Sentry Mode clips:

“Sentry Mode video clips are now saved in a separate folder on your USB drive to make them easier to review and manage. Also, the older Sentry mode video clips will now be automatically deleted if there is not enough space on the USB drive and Sentry Mode clips are using more than 5 Gigabytes of space.”

Not sure this impacts teslausb code or not.

@MikeBishop
Copy link

As described, this shouldn't matter for an instance that uploads regularly. If there were some reserved space to tuck recordings aside, you could prevent the Tesla from deleting them, but if that space were visible to the Tesla, you wouldn't hit the "not enough space" condition until you'd filled it anyway.

All in all, this looks like a no-op unless you have a smarter way to decide what to delete when full than "oldest."

@marcone
Copy link
Owner

marcone commented Sep 15, 2019

The "now saved in a separate folder" bit might indicate some path(s) need to be updated in Teslausb.

@sottile
Copy link

sottile commented Sep 18, 2019

Yup, indeed this archive loop function needs to be updated to search BOTH SavedClips and SentryClips

function archive_teslacam_clips () {
...
DIR_COUNT=$(find "$CAM_MOUNT"/TeslaCam/SavedClips/* -maxdepth 0 -type d | wc -l)
FILE_COUNT=$(find "$CAM_MOUNT"/TeslaCam/SavedClips/* -type f | wc -l)
...

and in archive_clips

It's probably worth it to future proof this and only EXCLUDE the "RecentClips" and the loop through all other directories and push the files.

Side note, using /* can be problematic. I can see that it is used in a lot of places. If there are a LOT of files, you can exceed the max command line with the glob expansion. In this case, it's probably better to eliminate the /* in both commands as they aren't necessary in this case. If you're trying to make sure it's a directory, using /. is better as in
find "$CAM_MOUNT"/TeslaCam/SavedClips/.
(and so on)

@SemoTech
Copy link

Thank you @sottile & @marcone

Do you know if these changes have already been added to the code, so I can update my Pi?
I am running early access V10 (2019.32.2.11) on my TM3 and this change has already occurred for me.

Appreciated!

@marcone
Copy link
Owner

marcone commented Sep 18, 2019

No, not yet

@SemoTech
Copy link

No, not yet

No rush @marcone but will be very excited once its' done. Will check back here....

@txex95
Copy link

txex95 commented Sep 18, 2019

Might also want to keep the folder structure of the transferred files to differentiate the Saved vs Sentry clips in the destination share.

@txex95
Copy link

txex95 commented Sep 22, 2019

Not very experienced programming. But willing to give it a shot. Does the SavedClips lines just need to be duplicated for SentryClips to capture both or is there more to it? Otherwise SD card is filling up with sentry clips.

@acraigl
Copy link

acraigl commented Sep 24, 2019

USB Sync to my NAS hasn't happened since I got v10. Only discovered it (not working) yesterday. Nothing uploaded since 9/21. I'll reflash and see if that changes anything in case something was corrupted during the firmware update.

@acraigl
Copy link

acraigl commented Sep 26, 2019

USB Sync to my NAS hasn't happened since I got v10. Only discovered it (not working) yesterday. Nothing uploaded since 9/21. I'll reflash and see if that changes anything in case something was corrupted during the firmware update.

Just posting back -- reflashed (latest beta, FWIW) and confirmed I have video files recording. Can SSH to the device no problem, but archiving isn't happening. When I tried to manually kick off the archiveloop script, it said it was already running. :|

@marcone
Copy link
Owner

marcone commented Sep 26, 2019

The current archive scripts don't take into account the new "SentryClips" folder. You can try the "v10" branch where I added some (untested, since I don't have v10 yet) code to archive SentryClips too.

@acraigl
Copy link

acraigl commented Sep 27, 2019

The current archive scripts don't take into account the new "SentryClips" folder. You can try the "v10" branch where I added some (untested, since I don't have v10 yet) code to archive SentryClips too.

Thanks. Thinks are still a bit wonky. Like, I only see the dashcam icon on my screen when I return home. ??? With v10 rolling to everyone now, I'm sure you'll work this out.

@SemoTech
Copy link

SemoTech commented Sep 28, 2019

Buster Release Image working perfectly following Tesla Firmware 2019.32.10.1 (v10 release)! All features confirmed functional.
Great job @marcone!

@acraigl
Copy link

acraigl commented Sep 28, 2019

Buster Release Image working perfectly following Tesla Firmware 2019.32.10.1 (v10 release)! All features confirmed functional.
Great job @marcone!

@SemoTech I wish I was so lucky. The dashcam icon never enables for me, and everything else seems to check out... can ssh to it, install completes successfully, etc. :( Can you post your config maybe? (redacting the personal info, obviously.)

@acraigl
Copy link

acraigl commented Sep 28, 2019

@marcone did a dmesg, assuming this is bad? Using the v10 branch with my original configuration file (from the june release)

[ 43.742612] No dialect specified on mount. Default has changed to a more secure dialect, SMB2.1 or later (e.g. SMB3), from CIFS (SMB1). To use the less secure SMB1 dialect to access old servers which do not support SMB3 (or SMB2.1) specify vers=1.0 on mount.
[ 43.957918] CIFS VFS: Error connecting to socket. Aborting operation.
[ 43.957951] CIFS VFS: cifs_mount failed w/return code = -101
[ 46.832409] loop0: p1
[ 97.414115] No dialect specified on mount. Default has changed to a more secure dialect, SMB2.1 or later (e.g. SMB3), from CIFS (SMB1). To use the less secure SMB1 dialect to access old servers which do not support SMB3 (or SMB2.1) specify vers=1.0 on mount.

FWIW, headless setup log said the archive was reachable....

@marcone
Copy link
Owner

marcone commented Sep 28, 2019

The "No dialect specified on mount" message doesn't matter I think (I see it too, and it doesn't prevent me from mounting the archive).
Maybe sudo mount /mnt/archive gives a more helpful error message.

@acraigl
Copy link

acraigl commented Sep 28, 2019

The "No dialect specified on mount" message doesn't matter I think (I see it too, and it doesn't prevent me from mounting the archive).
Maybe sudo mount /mnt/archive gives a more helpful error message.

It might but the bigger problem is I can't get the dashcam icon to come up anymore in the car. So although it appears to go through the archiveloop process, it doesn't have anything to copy anyway. I seem to have reduced/removed any dmesg errors by using the latest stretch image with the v10 archive and CIFS 3.0 setting... I just need to figure out why the damn thing isn't coming up in the car properly. :(

@acraigl
Copy link

acraigl commented Sep 28, 2019

@marcone ok, so after a lot of trial and error, I've found the reason that my icon wasn't appearing was because of a bad splitter. When I go direct it records no problem. Probably the act of getting v10 and moving stuff around jostled it just enough to stop working. Or something. Ugh.

Anyway, I'm still seeing the CIF error in dmesg, attached. And nothing is getting archived. Not sure what else to try at this point. Open to any suggestions.
dmseg20190928.txt

@marcone
Copy link
Owner

marcone commented Sep 28, 2019

diagnostics might show more: setup-teslausb diagnose

@acraigl
Copy link

acraigl commented Sep 28, 2019

output attached. I don't see anything out of the ordinary?
diagnose.txt

@marcone
Copy link
Owner

marcone commented Sep 28, 2019

Those diagnostics are truncated (by a lot).
Can you run setup-teslausb selfupdate and then setup-teslausb diagnose again?

@gacevedo
Copy link

Does the software update automatically every time the pi boots up, or do I need to SSH into it and run an update command of sorts, or do I need to go through the whole setup process again? I got v10 yesterday, and I'd like to test the changes. Thank you.

@normalfaults
Copy link

update process is here: https://github.com/marcone/teslausb/wiki

@acraigl
Copy link

acraigl commented Sep 28, 2019

Those diagnostics are truncated (by a lot).
Can you run setup-teslausb selfupdate and then setup-teslausb diagnose again?

Looks like everything was in a read-only mode for some reason. I did a mount -o remount, rw / and just ran setup-teslausb. That got everything cooking. Unfortunately the diagnose output looks exactly the same. 😢

Update: Was able to run the remountfs_rw script as root (as described in the troubleshooting doc) and reran. Updated diagnose output attached.
diagnose2.txt

@marcone
Copy link
Owner

marcone commented Sep 28, 2019

You probably need to set the branch specified in /root/teslausb_setup_variables.conf back to main-dev, because it looks like you're on another, older branch.

@acraigl
Copy link

acraigl commented Sep 28, 2019

You probably need to set the branch specified in /root/teslausb_setup_variables.conf back to main-dev, because it looks like you're on another, older branch.

OK, will update/rerun and let you know. Will main allow for the v10 video files?

@gacevedo
Copy link

So, got home after my first ride using v10, and after updating the Pi with instructions provided on here: https://github.com/marcone/teslausb/wiki. Videos are not making it into my server, so I SSH'ed into the Pi and the logs stopped on this line:

Sun 29 Sep 10:12:51 PDT 2019: linking /backingfiles/snapshots/newsnap/mnt/TeslaCam/SentryClips/2019-09-29_09-31-36/2019-09-29_09-21-31-back.mp4

Nothing else after that, for the past 21 minutes. Anything else I can do to debug this issue?

Note: before v10, everything was working just fine.

Thank you!

@gacevedo
Copy link

gacevedo commented Sep 29, 2019

If I run /etc/rc.local, I get the following output (pasting just the last few lines):

rm: cannot remove '/backingfiles/snapshots/newsnap/mnt/.Spotlight-V100/Store-V1/VolumeConfig.plist': Read-only file system
rm: cannot remove '/backingfiles/snapshots/newsnap/mnt/.Spotlight-V100/VolumeConfiguration.plist': Read-only file system
rm: cannot remove '/backingfiles/snapshots/newsnap/mnt/.fseventsd/fseventsd-uuid': Read-only file system
/dev/loop0p1 on /backingfiles/snapshots/newsnap/mnt type vfat (ro,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro)
/dev/loop1p1 on /backingfiles/snapshots/newsnap/mnt type vfat (ro,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro)
/dev/loop2p1 on /backingfiles/snapshots/newsnap/mnt type vfat (ro,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro)
/dev/loop3p1 on /backingfiles/snapshots/newsnap/mnt type vfat (ro,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro)
snapshot already mounted
fsck from util-linux 2.29.2
fsck.fat 4.1 (2017-01-24)
0x41: Dirty bit is set. Fs was not properly unmounted and some data may be corrupt.
Automatically removing dirty bit.
Performing changes.
/dev/loop4p1: 493 files, 646970/1420449 clusters
stat: cannot stat '/backingfiles/snapshots/newsnap/mnt/TeslaCam/SavedClips//': No such file or directory
/root/bin/make_snapshot.sh: line 60: collection: unbound variable

@gacevedo
Copy link

I ran...

sudo -i
/root/bin/remountfs_rw
/root/bin/setup-teslausb selfupdate
/root/bin/setup-teslausb
reboot

And it seems to be working again. No idea what the issue was. Hopefully this was a permanent fix. Will find out next time I take the car out.

@miles267
Copy link

miles267 commented Sep 29, 2019

@gacevedo - so you have teslausb working successfully with v10? Haven’t seen anyone confirm that here yet.

@jbienz
Copy link

jbienz commented Oct 15, 2019

@jbienz 12.2 is the car firmware
Oh. I see. I've had 2019.32.12.2 for 4 days now and it didn't make a difference for me. I'm going to try the vm.* caching settings from the other "too slow" issue and start following that thread. Thanks for all the help.

@miles267
Copy link

@jbienz what vm caching settings?

@jbienz
Copy link

jbienz commented Oct 17, 2019

I completely rebuilt my Pi last night as described in this post.

After the rebuild, I've noticed that Tesla seems to have only created two folders in the /mnt/cam folder:

RecentClips and SentryClips

I do not see any SavedClips folder as is mentioned in archiveloop line 279 and line 280.

Line 279 and 280 also do not do anything with the RecentClips folder.

@sottile said something in his message on 9/17 about ignoring RecentClips, but why?

I finally have teslausb working again, but none of the videos are ending up on my server because they're all in the RecentClips folder.

This seems like a bug. What am I missing here?

@japhule
Copy link

japhule commented Oct 17, 2019

The car only cares about having an existing TeslaCam folder. If the other folders don't exist, it'll create them for you.

@jbienz
Copy link

jbienz commented Oct 17, 2019

I think you're missing my point @japhule. I did not create any folders. The Car created RecentClips and SentryClips. The car has NOT created SavedClips.

The TeslaUSB archiveloop script is looking for SentryClips and SavedClips but the car created SentryClips and RecentClips.

TeslaUSB does not appear to be monitoring the same folders the car is creating.

@japhule
Copy link

japhule commented Oct 17, 2019

Did you save any footage by tapping the dashcam icon in the car to save some footage that would go into the saved clips folder?

@jbienz
Copy link

jbienz commented Oct 17, 2019

No. Not yet. And that could explain why there is no SavedClips folder. However, why is TeslaUSB no longer monitoring or saving RecentClips? I see in the forums all the time "I was so scared at the time that I forgot to tap the save icon".

@japhule
Copy link

japhule commented Oct 17, 2019

Tesla USB by default only syncs the saved clips and sentry clips folders. I believe there are additional modifications or settings to synch the recent clips. I don't care to capture recent clips personally but it is at least accessible via the smb function.

@miles267
Copy link

Tesla USB by default only syncs the saved clips and sentry clips folders. I believe there are additional modifications or settings to synch the recent clips. I don't care to capture recent clips personally but it is at least accessible via the smb function.

I have both SentryClips and SavedClips on my CIFS file share, however there's never anything uploaded to SavedClips. Also I've added to archive-clips.sh the sync of RecentClips, but no such folder or contents has yet been uploaded to the file share either.

@jbienz
Copy link

jbienz commented Oct 17, 2019

@marcone I would like to have an option to sync all clips, including recent ones. I don't see an option for this in the archivelooop. Would it be possible to add another export option to include RecentClips as well?

@jbienz
Copy link

jbienz commented Oct 17, 2019

@miles267 I don't think you'd edit this in archive-clips.sh, I think you'd need to add it to archiveloop. Specifically on lines 279 and 280. Or maybe you actually need to do it in both places. But I'm hoping @marcone can confirm what changes would need to be made and possibly even add this as an option.

@miles267
Copy link

@miles267 I don't think you'd edit this in archive-clips.sh, I think you'd need to add it to archiveloop. Specifically on lines 279 and 280. Or maybe you actually need to do it in both places. But I'm hoping @marcone can confirm what changes would need to be made and possibly even add this as an option.

Thanks @jbienz. It would be absolutely awesome if @marcone could add this. By no means do I trust myself well enough to do it correctly.

@marcone
Copy link
Owner

marcone commented Oct 17, 2019

No. Not yet. And that could explain why there is no SavedClips folder. However, why is TeslaUSB no longer monitoring or saving RecentClips?

It never did.

@jbienz
Copy link

jbienz commented Oct 17, 2019

It never did.

I understand that folder didn't exist before. But that's not what I meant. Let me try rewording this.

Before V10 teslausb backed up all videos recorded by the car. After V10, teslausb only backs up sentry mode events and triggered recordings.

Why does teslausb no longer backup all videos? And would it be possible to get this feature back please?

@prathiraj
Copy link

teslausb never backed up RecentClips folder. The videos is the RecentClips folder are rotated every hour by the car. @marcone has a feature to capture the rotated videos through snapshots (default feature). But they are also rotated when space runs out and never backed up. Hope this explains.

@jbienz
Copy link

jbienz commented Oct 17, 2019

Ahhh. Thank you @prathiraj for that explanation, that's a huge help in understanding what's going on.

I only recently added SNAPSHOTS_ENABLED=false to my config file in hopes of stopping the "too slow" errors. Would turning snapshots back on give me back all clips? I never fully understood what I was disabling, I just knew that many people were disabling it in an attempt to solve "too slow".

@prathiraj
Copy link

SNAPSHOTS_ENABLED=true will enable the snapshots feature, but still they will not be backed up through archiveloop. You need to access them through samba.

@jbienz
Copy link

jbienz commented Oct 17, 2019

Okay. So, what changed before and after V10 then? Before V10 I received hours and hours of driving footage to my server, whether I hit the "save now" button or not. What changed?

@miles267
Copy link

miles267 commented Oct 17, 2019

How do you enable Samba? I’ve not yet done this. Do I need to have both:
export SAMBA_ENABLED=true AND export SAMBA_GUEST=true?

Or when you have SAMBA enabled, and data is transferred to CIFS, does it defeat the purpose as the data is transferred off the RP0 to the CIFS file share and there's nothing to view over SAMBA?

@marcone
Copy link
Owner

marcone commented Oct 17, 2019

It never did.

I understand that folder didn't exist before.

The RecentClips folder existed before v10 (it was introduced in v9, and prior to that the recordings where directly under the TeslaCam folder)

But that's not what I meant. Let me try rewording this.

Before V10 teslausb backed up all videos recorded by the car.

No it didn't. It never backed up the RecentClips folder, nor did it ever back up any non-sentry, non-saved recordings when they were still directly under TeslaCam.

After V10, teslausb only backs up sentry mode events and triggered recordings.

Why does teslausb no longer backup all videos? And would it be possible to get this feature back please?

It never did, so there's no "back". As to why it doesn't: it's because I don't find it very useful to only back up the most recent hour. With snapshots, it's possible to archive everything, but I haven't gotten around to that yet.

@jbienz
Copy link

jbienz commented Oct 18, 2019

It never did, so there's no "back".

I am obviously misunderstanding something really important and I do sincerely apologize for that.

I went back and looked more carefully at my history. As recent as 2019-10-02 I have videos of me just driving. Not Sentry (obviously, since I'm driving) and not manually triggered either. Just videos of me driving. After 2019-10-02 I can't find any more videos like that.

Again, I apologize. I thought the last time I saw these driving videos was before I updated the car to V10. I didn't realize there were videos like that as recently as 10-02. I honestly don't understand why these videos stopped after 10-02. I didn't set the SNAPSHOTS_ENABLED=false flag until 10-12 or maybe 10-13.

Really sorry if I frustrated you @marcone. That wasn't my intention. I'm just trying to understand where all of these driving videos went and how I might be able to get them back. My fear is if something ever happens to me while driving and I'm so shook up about it that I forget to hit the "save now" button. I have lots of space on my server, so I'd rather save it all and just delete old files whenever I feel like cleaning up space.

Thanks for all your hard work. It really is appreciated.

@miles267
Copy link

@jbienz now that you mention this, I’ve not seen those videos either with V10 since I’ve moved from a local USB Samsung T5 SSD to Teslausb. It hasn’t occurred to me until I read your post. Where are those videos saved?

@japhule
Copy link

japhule commented Oct 18, 2019

Some of my sentry captures show me driving and then parking at the end when it captures an event. Maybe some of those are like that.

@marcone
Copy link
Owner

marcone commented Oct 18, 2019

I went back and looked more carefully at my history. As recent as 2019-10-02 I have videos of me just driving. Not Sentry (obviously, since I'm driving) and not manually triggered either. Just videos of me driving. After 2019-10-02 I can't find any more videos like that.

The only things I can think of to explain that are that either you had made a local modification so that it also saved RecentClips (and then ran setup-teslausb on 10-02, overwriting your modification), or that the videos of you driving are all within 10 minutes of you parking and enabling Sentry mode and immediately getting a Sentry event (which save the previous 10 minutes, including recordings before you enabled Sentry mode).

Really sorry if I frustrated you @marcone. That wasn't my intention. I'm just trying to understand where all of these driving videos went and how I might be able to get them back. My fear is if something ever happens to me while driving and I'm so shook up about it that I forget to hit the "save now" button. I have lots of space on my server, so I'd rather save it all and just delete old files whenever I feel like cleaning up space.

In that case saving RecentClips will only help if you go home within an hour of being shook up, since the car deletes everything older than an hour. Encounter something noteworthy during your morning commute, and that footage will be gone by the time you get home at night. Snapshots largely solve that problem.

@jbienz
Copy link

jbienz commented Oct 18, 2019

you had made a local modification so that it also saved RecentClips

Do you mean to the teslausb scripts? If so, no I have never modified those.

or that the videos of you driving are all within 10 minutes of you parking and enabling Sentry mode

I'm not sure that's the case either, but let me explain why. For example, the clip on 10-02 starts with me leaving a parking lot at 18:38 and it ends with the car in the garage at 18:48. There is no motion in the last clip with the car in the garage, and I have sentry mode disabled at my home location. Over the 10 minutes between the parking lot and the garage there were 10 clips recorded for each camera with a total of 40 clips. This was after I'd upgraded to V10 so it included the rear camera.

The last one with driving before that was on 09-21. It starts at 15:02 with me sitting at a stop light. It ends at 15:12 with me getting close to the house but stops before I actually pull into the driveway. There were 11 clips recorded for each camera with a total of 33 clips. This was before I installed V10 so it did not include the rear camera.

Snapshots largely solve that problem.

Enabling snapshots should solve that now? Or it will when you get around to archiving everything?

@jbienz
Copy link

jbienz commented Oct 18, 2019

Some of my sentry captures show me driving and then parking at the end when it captures an event. Maybe some of those are like that.

Sorry, I missed this comment before. Has it ever happened when you've parked at a location where Sentry is disabled? If so, we may have a correlation.

@jbienz
Copy link

jbienz commented Oct 18, 2019

One other thing that just popped into my head … I do have homelink and I use it. The garage door open button does pop up fairly close to the TeslaCam "save now button". It's possible that in those two scenarios maybe I accidentally tapped on the "save now" button when I went to open the garage door and just didn't realize it?

@marcone
Copy link
Owner

marcone commented Oct 18, 2019

Enabling snapshots should solve that now? Or it will when you get around to archiving everything?

Yes, though until then you can access the extended RecentClips via Samba.

One other thing that just popped into my head … I do have homelink and I use it. The garage door open button does pop up fairly close to the TeslaCam "save now button". It's possible that in those two scenarios maybe I accidentally tapped on the "save now" button when I went to open the garage door and just didn't realize it?

That certainly seems plausible. If that's a regular occurrence for you, then you probably have SavedClips entries now of you arriving home and opening the garage.

@marcone
Copy link
Owner

marcone commented Oct 25, 2019

I think this issue has served its purpose. v10's directory structure and additional recording file is now supported

@marcone marcone closed this as completed Oct 25, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests