I did that first but that always required much more resources than doing it yourself because every docker starts it’s own database and it’s own nginx/apache server in addition to the software itself.
Now I have just one Postgresql database instance running with many users and databases on it. Also just one Nginx which does all the virtual host stuff in one central place. And both the things which I install with apt and manually are set up similarly.
I use one docker setup for firefox-sync but only because doing it manually is not documented and even the docker way I had to research for quite some time.
What? No it doesn’t… You could still have just one postgresql database if you wanted just one. It is a big antithetical to microservices, but there is no reason you can do it.
But then you can’t just use the containers provided by the service developers and have to figure out how to redo their container which in the end is more work than just run it manually.
I have very rarely ran into such issues. can you give an example of something that works like that? it sounds to be very half-assed by the developer. only pihole comes to mind right now (except for the db part, because I think it uses sqlite)
Typically, the container image maintainer will provide environment variables which can override the database connection. This isn’t always the case but usually it’s as simple as updating those and ensuring network access between your containers.
all of these run the database in a separate container, not inside the app container. the latter would be hard to fix, but the first is just that way to make documentation easier, to be able to give you a single compose file that is also functional in itself. none of them use their own builds of the database server (though lemmy with its postgres variant may be a bit of an outlier), so they are relatively easy to configure for an existing db server.
all I do in cases like this is look up the database initialization command (in the docker compose file), run that in my primary postgres container, create a new docker network and attach it to the postgres stack and the new app’s stack (stack: the container composition defindd by the docker compose file). and then I tell the app container, usually through envvars or command line parameters embedded in the compose file, that the database server is at xy hostname, and docker’s internal DNS server will know that for xy hostname it should return the IP address of the container that is named xy, through the appropriate docker network. and also the user and pass for connection. from then, from the app’s point of view, my database server in that other container is just like a dedicated physical postgres machine you put on the network with its own cable going to a switch.
unless very special circumstances, where the app needs a custom build of postgres, they can share a single instance just fine. but in that case you would have to run 2 postgres instances even without docker, or migrate to the modified postgres, which is an option with docker too.
You absolutely can. It’s not like the developers of postgresql maintain a version of postgresql that only allows one db. You can connect to that db and add however many things you want to it.
Yeah why wouldn’t you want to know how things work!
I obviously don’t know you, but to me it seems that a majority of Docker users know how to spin up a container, but have zero knowledge of how to fix issues within their containers, or to create their own for their custom needs.
That’s half the point of the container… You let an expert set it up so you don’t have to know it on that level. You can manage fast more containers this way.
And I have no troubling spinning up new services, fast. Currently sitting at around ~30 Internet-facing services, 0 docker containers, and reproducing those installs from scratch + restoring backups would be a single command plus waiting 5 minutes.
Fair, but others, unless they are getting paid for it, just want their shit to work. Same as people who take their cars to a mechanic instead of wrenching on it themselves, or calling a handyman when stuff breaks at home. There’s nothing wrong with that.
I literally get paid to do this type of work and there is no way for me to be an expert in all the services that our platform runs. Again, that’s kind of the point. Let the person who writes the container be the expert. I’ll provide the platform, the maintenance, upgrades, etc… the developer can provide the expertise in their app.
A lot of times it is necessary to build the container oneself, e.g., to fix a bug, satisfy a security requirement, or because the container as-built just isn’t compatible with the environment. So in that case would you contract an expert to rebuild it, host it on a VM, look for a different solution, or something else?
Containerfiles are super easy to write. For the most part if you can do it in a VM, you can do it in a container. This sort of thing is why you would move to containers. Instead of being the “expert” in all the apps you run, you can focus on the things that actually need your attention.
It’s not like it’s so hard to rebuild a container for the occasional services that needs it. but it’s still much better than needing to do it with every single service
It depends on the container I suppose. There are some that are very difficult to rebuild depending on what’s in it and what it does. Some very complex software can be ran in containers.
I’ve been wanting to tinker with NixOS. I’ve stuck in the stone ages automating VM deployments on my Proxmox cluster using ansible. One line and about 30 minutes (cuda install is a beast) to build a reproducible VM running llama.cpp with llama-swap.
30, that’s cute. I currently have 70 containers running on my home server. That doesn’t include any lab I run or the stuff I use at work. Containers make life much easier. I also guarantee you don’t know those apps as well as you think you do either. Just being able to install and configure something doesn’t mean you know the inner workings of them. I used to do the same thing you do. Eventually, I would rather spend my time doing other things or learning certain things more in-depth and be okay with a working knowledge of others. It can be fun and rewarding to do things the hard way but don’t kid yourself and think you’re somehow superior for doing it that way.
Correct, not all containers are for services. I would never say that docker is superior. I would however say that containers are (I can be pedantic too). They’re version-controlled, they come with the correct dependencies, etc… There are many reasons why developing with containers is superior and I’m sure you’re aware of them already. Everyone is moving to do exactly that. There are always edge cases, but those are few and far between these days.
I use apps on my phone, but have no clue how to troubleshoot them. I have programs on my computer that I hardly know how to use, let alone know the inner workings of. How is running things in Docker any different? Why put down people who have an interest in running things themselves?
I know you’re just trying to answer the above question of “why do it the hard way”, but it struck me as a little condescending. Sorry if I’m reading too much into it!
No, I actually think that is a good analogy. If you just want to have something up and running and use it, that’s obviously totally fine and valid, and a good use-case of Docker.
What I take issue with is the attitude which the person I replied to exhibits, the “why would anyone not use docker”.
I find that to be a very weird reaction to people doing bare metal. But also I am biased. ~30 Internet facing services, 0 docker in use 😄
This is interesting to me. I run all of my services, custom and otherwise, in docker. For my day job, I am the sole maintainer of all of our docker environment and I build and deploy internal applications to custom docker containers and maintain all of the network routing and server architecture. After years of hosting on bare metal, I don’t know if I could go back to the occasional dependency hell that is hosting a ton of apps at the same time. It is just too nice not having to think about what version of X software I am on and to make sure there isn’t incompatibility. Just managing a CI/CD workflow on bare metal makes me shudder.
Not to say that either way is wrong, if it works it works imo. But, it is just a viewpoint that counters my own biases.
Why wouldn’t you just use Docker or Podman
Manually installing stuff is actually harder in a lot of cases
I did that first but that always required much more resources than doing it yourself because every docker starts it’s own database and it’s own nginx/apache server in addition to the software itself.
Now I have just one Postgresql database instance running with many users and databases on it. Also just one Nginx which does all the virtual host stuff in one central place. And both the things which I install with apt and manually are set up similarly.
I use one docker setup for firefox-sync but only because doing it manually is not documented and even the docker way I had to research for quite some time.
What? No it doesn’t… You could still have just one postgresql database if you wanted just one. It is a big antithetical to microservices, but there is no reason you can do it.
But then you can’t just use the containers provided by the service developers and have to figure out how to redo their container which in the end is more work than just run it manually.
I have very rarely ran into such issues. can you give an example of something that works like that? it sounds to be very half-assed by the developer. only pihole comes to mind right now (except for the db part, because I think it uses sqlite)
edit: I now see your examples
Typically, the container image maintainer will provide environment variables which can override the database connection. This isn’t always the case but usually it’s as simple as updating those and ensuring network access between your containers.
Some examples:
and many more.
all of these run the database in a separate container, not inside the app container. the latter would be hard to fix, but the first is just that way to make documentation easier, to be able to give you a single compose file that is also functional in itself. none of them use their own builds of the database server (though lemmy with its postgres variant may be a bit of an outlier), so they are relatively easy to configure for an existing db server.
all I do in cases like this is look up the database initialization command (in the docker compose file), run that in my primary postgres container, create a new docker network and attach it to the postgres stack and the new app’s stack (stack: the container composition defindd by the docker compose file). and then I tell the app container, usually through envvars or command line parameters embedded in the compose file, that the database server is at xy hostname, and docker’s internal DNS server will know that for xy hostname it should return the IP address of the container that is named xy, through the appropriate docker network. and also the user and pass for connection. from then, from the app’s point of view, my database server in that other container is just like a dedicated physical postgres machine you put on the network with its own cable going to a switch.
unless very special circumstances, where the app needs a custom build of postgres, they can share a single instance just fine. but in that case you would have to run 2 postgres instances even without docker, or migrate to the modified postgres, which is an option with docker too.
Well, yes that’s best practice. That doesn’t mean you have to do it that way.
You absolutely can. It’s not like the developers of postgresql maintain a version of postgresql that only allows one db. You can connect to that db and add however many things you want to it.
Yeah why wouldn’t you want to know how things work!
I obviously don’t know you, but to me it seems that a majority of Docker users know how to spin up a container, but have zero knowledge of how to fix issues within their containers, or to create their own for their custom needs.
That’s half the point of the container… You let an expert set it up so you don’t have to know it on that level. You can manage fast more containers this way.
OK, but I’d rather be the expert.
And I have no troubling spinning up new services, fast. Currently sitting at around ~30 Internet-facing services, 0 docker containers, and reproducing those installs from scratch + restoring backups would be a single command plus waiting 5 minutes.
Fair, but others, unless they are getting paid for it, just want their shit to work. Same as people who take their cars to a mechanic instead of wrenching on it themselves, or calling a handyman when stuff breaks at home. There’s nothing wrong with that.
I literally get paid to do this type of work and there is no way for me to be an expert in all the services that our platform runs. Again, that’s kind of the point. Let the person who writes the container be the expert. I’ll provide the platform, the maintenance, upgrades, etc… the developer can provide the expertise in their app.
A lot of times it is necessary to build the container oneself, e.g., to fix a bug, satisfy a security requirement, or because the container as-built just isn’t compatible with the environment. So in that case would you contract an expert to rebuild it, host it on a VM, look for a different solution, or something else?
Containerfiles are super easy to write. For the most part if you can do it in a VM, you can do it in a container. This sort of thing is why you would move to containers. Instead of being the “expert” in all the apps you run, you can focus on the things that actually need your attention.
It’s not like it’s so hard to rebuild a container for the occasional services that needs it. but it’s still much better than needing to do it with every single service
It depends on the container I suppose. There are some that are very difficult to rebuild depending on what’s in it and what it does. Some very complex software can be ran in containers.
Is that with Ansible or your own tooling or something else?
NixOS :)
Maybe I should have clarified that liking bare-metal does not imply disliking abstraction
I’ve been wanting to tinker with NixOS. I’ve stuck in the stone ages automating VM deployments on my Proxmox cluster using ansible. One line and about 30 minutes (cuda install is a beast) to build a reproducible VM running llama.cpp with llama-swap.
30, that’s cute. I currently have 70 containers running on my home server. That doesn’t include any lab I run or the stuff I use at work. Containers make life much easier. I also guarantee you don’t know those apps as well as you think you do either. Just being able to install and configure something doesn’t mean you know the inner workings of them. I used to do the same thing you do. Eventually, I would rather spend my time doing other things or learning certain things more in-depth and be okay with a working knowledge of others. It can be fun and rewarding to do things the hard way but don’t kid yourself and think you’re somehow superior for doing it that way.
Containers != services.
I don’t think I am better than anyone. I jumped into these comments because docker was pushed as superior, unprompted.
Installing and configuring does not an expert make, agreed; but that’s not what I said.
I would say I’m pretty knowledgeable about the things I host though, seeing as I am a contributor and / or package maintainer for a number of them…
Correct, not all containers are for services. I would never say that docker is superior. I would however say that containers are (I can be pedantic too). They’re version-controlled, they come with the correct dependencies, etc… There are many reasons why developing with containers is superior and I’m sure you’re aware of them already. Everyone is moving to do exactly that. There are always edge cases, but those are few and far between these days.
You can customize or build custom containers with a Dockerfile
Also, I want to know how containers work. That’s way more useful.
I use apps on my phone, but have no clue how to troubleshoot them. I have programs on my computer that I hardly know how to use, let alone know the inner workings of. How is running things in Docker any different? Why put down people who have an interest in running things themselves?
I know you’re just trying to answer the above question of “why do it the hard way”, but it struck me as a little condescending. Sorry if I’m reading too much into it!
No, I actually think that is a good analogy. If you just want to have something up and running and use it, that’s obviously totally fine and valid, and a good use-case of Docker.
What I take issue with is the attitude which the person I replied to exhibits, the “why would anyone not use docker”.
I find that to be a very weird reaction to people doing bare metal. But also I am biased. ~30 Internet facing services, 0 docker in use 😄
This is interesting to me. I run all of my services, custom and otherwise, in docker. For my day job, I am the sole maintainer of all of our docker environment and I build and deploy internal applications to custom docker containers and maintain all of the network routing and server architecture. After years of hosting on bare metal, I don’t know if I could go back to the occasional dependency hell that is hosting a ton of apps at the same time. It is just too nice not having to think about what version of X software I am on and to make sure there isn’t incompatibility. Just managing a CI/CD workflow on bare metal makes me shudder.
Not to say that either way is wrong, if it works it works imo. But, it is just a viewpoint that counters my own biases.