Videos are downloaded, but they are nowhere #47
Comments
|
Hey Jana, The path on your machine for the second volume in streamsphere and the volume in content require to be the same! I have made the fix in the docker compose below. Please try it out and let me know if this works for you? :-) for the downloaded media, based on how yt-dlp works, it is likely that the downloaded videos should have been in /app/cmd which was not mounted to a permanent volume and should have been deleted when the running streamsphere container stopped. I would also suggest that you remove the existing docker volumes and, containers and start afresh! Thanks for providing the detailed logs and apologies for the late reply! |
|
Hi, but now the only thing that has changed is that I see the thumbnails while downloading (I didn't see any previously), but all that physically appears in the folder are the thumbnails. The app clearly shows the megabytes of the video downloading, but only a thumb is stored. BTW, It's Jan ;-) |
|
Hey Jan, :) thanks for the update! It seems like one error is fixed and thumbs are going into the right location. I really appreciate your time and efforts on this! :-) Update: If the video files are also being stored in the content directory, kindly try the following - for the error that we have now, it looks like the database is not in place for the application, although it is shipped initialized within the docker image itself. This could be because the bind-mount did not bind correctly onto the physical location.
Once the database is in place, do try to download through the streamsphere now and it should work! |
|
No, as of now, no video is to be found anywhere, only the thumbnails are stored into
but it is 0 Bytes in size, so nothing seems to be saved to it after job is done. It's been created by the app as soon as it started. Should I do the steps anyway? |
|
No no, pls dont perform the steps since the video files are not in place. Can you pls try to remove the streamsphere & streamsphere-content containers, their images, and the docker bind-mounts as well and then try again with the "docker compose up -d". The path for the videos is in the golang binary file, the videos should be downloaded into the /content/youtube.com/[name of playlist]/Videos/ directory. Also just to confirm you are using an AMD64 machine or ARM64? |
|
Ok, I have erased everything, including images, containersm, etc and redeployed from scratch. and the app doesn't move from there, nor I can open any UI. Tried to restart, but it just repeated the same behavior. |
|
Hi Jan, This is quite unexpected! let me get back to you in sometime, I will make an omv6 + portainer installation and try to replicate the issue you are facing. Sorry for the troubles! and I really appreciate your time and effort on this! :) |
|
Hey, If you can make just one last try, and I understand its asking a bit too much but can you pls remove container, bind mount, images and once try with the following docker-compose: Thank you! If you're not able to do this test I will totally understand! |
|
Hey, no worries! Update: So, I've just updated the docker compose to use named volumes instead of bind mounts.. |
|
Yes yes, noticed soon after posting. Ok, tried this, but something wrong happened after the last time I deleted everything. I cannot open UI: And the app is stuck to the above picture. I have deployed your compose as-is, untouched. EDIT |
|
Owh nice! :) Atleast we are where we were before! |
|
Ok, I have downloaded a couple of videos and I can see them from Videos UI tab. EDIT Maybe Streamsphere needs to assign rights to /content folder? |
|
I can totally understand, this effort just helps us understand the issue! and as I see your edit, I agree, this definitely has something to do with rights. Let me check on this and get back to you. :) Thank you! |
|
No problem, but I'll be away for a few hours, cheers. |
|
No worries, pls reply back with your convenience! Cheers! |
|
Hi Jan, Please stop and remove the volumes and containers, keeping the images should be fine though, after that pls try setting the rights to the directories on host machine to 666 with below commands - chmod 666 /srv/dev-disk-by-uuid-5b67514d-485e-4306-873e-b1cbb54ccf99/Config/streamsphere/db-data -R start the containers with docker compose up at this point. Please let me know if this fixes your problem! |
rs-anantmishra
added
help wanted
|
Done, but as before videos are nowhere to be found, despite having bind the folders. Also this time videos are not even present in the Videos UI tab. |
|
Hey Jan, That is definitely worth a try!! Do let me know if that works!! Thank You! |
|
I have added PUID and PGID env, but unfortunately this produced no effect, no video is visible in any folder, nor in the Videos UI tab. This is the log Going offline. |
|
Wow! Let me reproduce and sort this on an omv setup! I'll get back to you! |
|
Hey Jan, I tried a couple of combinations with bind mounts unfortunately it never reproduces for me, just wanted to give you an update. This is something very specific, but it clearly has to do with permissions for the docker container. Thanks! |
|
Did you try with PUID:PGID? |
|
I think the network is alright! I did try with 998/100 and 1000/1000 works perfectly with both, tried a bunch of different combinations of permissions as well, Still no luck, it always worked correctly, I'm not really sure what we could be missing at this point! :/ |
|
Give me some more time, I will research this more, and get back to you! |
|
Don't worry at all, take your time, I'm not a programmer, so I can only make tests. |
|
So far I reproduced the bug and did a dirty fix on my install. It only happens when binding a folder instead of using Docker volume (still need to figure out why). The dirty fix? Initialize db manually: Now everything works just fine. |
|
Hey Eidenz, are you also facing the same problem? are you also using OMV? |
|
I am using UNRAID on my end. As with all things that work in a volume but not with a binding, it is always a permission issue. Might want to specify a user. Might want to try directly with the user flag, like: services:
streamsphere:
user: "1000:1000" |
|
I can give a quick try, haven't started pouring my content in yet |
|
Hey @janaxhell , Could you pls try with instead of 998/100, if you're okay with that? |
Pls do try that!! |
|
I was unable to fix using that. In fact, it completely fails to create the file now. I believe something might need to be done on the Dockerfile or something. FYI, UNRAID and OMW uses special PUID/PGID, which might be why it works on a simple ubuntu but not those two. |
Hey @Eidenz, Thanks for making the test! Yes you're correct, it seems like it's something related to that! In shell terminal: cat /etc/passwd | grep $(whoami) Thanks!! |
|
That would be 0:0. Users work very differently in UNRAID, would need a read at the forums to understand what's going on there. And for correction on my previous statement, UNRAID itself doesn't use 99:100. It's just very commonly used for docker containers there. What I find weird is why the app is able to create the .db file but not initialize its content. It was clearly able to write to it after I did it manually. |
|
In that case, maybe a simple fix would be to do like a lot of apps and initialize in your app rather than the Dockerfile. Like, check for empty db / table not existing and initialize. |
|
The .db file gets created when the app tries to open it and since its sqlite, it would just create the db! Thanks for your inputs @Eidenz ! This really helps! :-) |
Hello anant, sorry, but I've been busy with a major problem with one of my servers. OT: I have a test next tuesday, so I won't be able to try anything on Streamsphere until then. |
|
Hey @janaxhell Thanks for trying, I will fix it through code Jan, all the best for your test!! x) |
|
Changes will be added to address this during application runtime.
|
|
Hi there, I have a similar problem and I think I found the problem The same thing happens in docker as well. If I explore the filesystem of streamsphere in docker-desktop, I can see /Videos appearing with the downloaded video inside. Unfortunately, mapping /Videos is not a solution as all videos from all channels end up in there |
|
After looking around the code and at the files I think the problem may be related to extracting or parsing MediaInfo (SavedInfo). When assembling the path for the file, the domain and channel names from the SavedInfo structure are used to create the directory structure and the video title for the filename. For the thumbnail, there seems to be no problem, but for the Video file, the saved path results in just /Videos and the final filename does not contain the video title, only the ID. |
|
Weird conclusion knowing that it works perfectly fine in a docker volume or when you manually create the database file... See my previous comment on the matter.
|
|
I took the database file from database/db from the repo and placed in in database/db and now it works. |
|
@toneck Hey, this was concluded to be a permissions issue. It did replicate for me about ~4 days back, this will be properly sorted in the upcoming release. :-) I do appreciate the time you have spent on the codebase, however, the bizarre outcomes are really because of permissions/db not being setup correctly for some reason when using bind mounts. |
I've just deployed Streamsphere, I've added the first random playlist YT proposed me and the UI actually showed the downloads were happening. But after finishing, both Playlists and Videos sections were empty. Also the content folder was empty. The only thing that actually happened was the video thumbnails stored in the config folder.
This is my compose, what's wrong?
This is the streamsphere-content log:
And this is the last bit of the main app log:
The text was updated successfully, but these errors were encountered: