It looks like your issue might stem from one of the following causes:

Possible Causes and Fixes:

1. Misconfiguration in Nginx Reverse Proxy

   • If Nginx is improperly handling WebSockets or HTTP requests, it could be modifying or breaking JSON responses from your Llama 2 API.
   • Fix: Ensure your Nginx configuration includes proper proxy settings:
     nginx

     CopyEdit

     location / {
     proxy_pass http://localhost:PORT; # Replace PORT with your Llama 2 API port
     proxy_set_header Host $host;
     proxy_set_header X-Real-IP $remote_addr;
     proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
     proxy_set_header X-Forwarded-Proto $scheme;

     proxy_buffering off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
}


2. Certbot HTTP-01 Challenge Interference

   • Certbot might have temporarily modified your Nginx config and is interfering with normal operation.
   • Fix: Try restarting Nginx after Certbot finishes:
     nginx

     CopyEdit

     sudo systemctl restart nginx


3. Mixed Content Issue (HTTP vs. HTTPS)

   • If the AI server is still using HTTP but Nginx is serving HTTPS, responses might be malformed.
   • Fix: Ensure the Llama 2 backend API is also running with HTTPS or use proxy_redirect:
     nginx

     CopyEdit

     proxy_redirect http:// https://;


4. JSON Response Encoding Issue

   • If Nginx is interfering with response headers, ensure JSON content-type is correctly set.
   • Fix: Add:
     nginx

     CopyEdit

     proxy_set_header Content-Type application/json;


Next Steps

• Check the Nginx logs:
  nginx

  CopyEdit

  sudo journalctl -u nginx --no-pager --lines=50

• Verify Certbot logs for potential misconfigurations:
  bash

  CopyEdit

  sudo cat /var/log/letsencrypt/letsencrypt.log

• Try disabling Nginx temporarily and directly accessing the AI server to confirm JSON responses are correct.

• Assumptions
  • You need to resize a drive without a GUI.
  • You have made some backup of your VM/Physical drive
• Notes
  • I got locked out of my Ubuntu Desktop GUI for this tutorial and have to putty to the machine as shown in the image below.
  • 
• List you disks
  • df -h
    • You will get a large output but you need to look for something similar to this
    • 
• Find the partition that is out of space. (mine was /dev/sda3/)
  • sudo fdisk /dev/sda
• Delete and remake the partition (YOU WILL NOT LOSE DATA), mine was partition 3
  • d
    • delete a partition
  • 3
    • Partition 3
  • n
    • Make a new partition
  • 3
    • Make it partition 3
  • PRESS ENTER TWICE
    • Specifying the starting and ending blocks. I want the default so I press enter for both questions.
  • Y
    • Yes, delete the signature
  • w
    • Write the changes.
  • 
  • Control + C
    • Exit fdisk
• Resize the file system
  • sudo resize2fs /dev/sda3
• Verify the changes were made
  • df -h
    • The new size of your file system will be shown.
  • 

• Assumptions
  • You have installed and configured docker using the documentation here.
• Install Docker Desktop
  • Docker Desktop is a Windows solution to having the docker engine on Windows 10/11 (THIS CAN NOT RUN ON WINDOWS SERVERS) and can be downloaded here.
  • After running the install, there will be a checkbox asking if you want to run it using WSL2, uncheck this as most Windows systems don’t have this downloaded by default and HyperV works just as well.
  • After installing and restarting, move on
• Install DockStation
  • This is a GUI for creating, managing, and deleting Docker containers on both remote and local Docker Hosts.
    • 
  • To install this, go the the web address here and download the Windows executable. Just run the .exe file and click next until the app is installed.
• DockStation Configuration
  • Now that you have DockStation installed, you need to connect it to your remote server.
  • To do this go to the “Containers” top and go to the top left and edit the machine it is connected to (changing it from localhost).
    • 
  • You will then add a new connection and log in via “SSH”. You will enter the IP address of your docker host and enter a username that has access to the docker application on that host (I am going to use root).
    • 
  • You then need to make sure you have the correct connection selected and click “connect” to connect to the Docker instance.
    • 
  • Once you log in, you will be able to view your docker machines, create new ones, connect to the shell of these machines, delete them, monitor them, and so on.
    • 

• Assumptions
  • You have powered down the machine (safely of course)
  • The server is unplugged and residual power has been drained.
• Find the broken drive
  • This is indicated via the red LED on the front of the drive bays.
    • FLASHING RED = HOT SPARE
    • SOLID RED = BROKEN/BAD HDD
      • 
  • Remove the drive from the drive bay and replace it with a drive of the EXACT same model. If it is a different model, it can not be added as a hot spare, only as an “Up Good” drive.
    • 
• Power on the Server
  • As the server powers on, it will conduct tests such as normal POST and gather information on the RAID controller. As the machine powers on, you will eventually come to the following screen after some minutes of booting.
    • 
  • Once this screen comes up, press “CTRL + R” to enter the raid controller settings. The screen will look like the following.
    • 
  • Once you get to this screen, press enter to enter the primary RAID controller settings.
• Configure the drive and add it as a Hot Spare
  • Once you are in the RAID controller, it will look somewhat like the following.
    • 
  • As you can see, there are only THREE hot swap drives. We need to add the new drive to act as the fourth spare.
  • To do this, tab over to “PD Mgmt” to see all of the disks. The screen will look as follows.
    • 
  • Here you can see a list of all drives connected to the RAID controller. The one in the”JBOD” state is going to be the drive that needs to be changed.
  • To do this, press “F2” while hovering the drive to bring up the options. You will then be presented with the following menu.
    • 
  • In this menu, you will select “Make Unconfigured Good” to change the drive to an “Up Good” state. You will then go back to the menu to make it a Hot Spare.
    • 
  • Once you are back to this menu, select “Make Global HS” to make it a Hot Spare for all virtual drives.
  • You can then go back to “VD Mgmt” and expand the “Hot Spare Drives” section to see your new hot spare!
    • 
• Reboot the Server
  • You can now reboot the machine and bring it back to full operation!

•  Assumptions
  • You have an Ubuntu 22.04 Linux server created
    • 4GB of RAM
    • 30GB of Drive Storage
• Install Docker
  • sudo apt-get update
  • sudo apt-get install ca-certificates curl gnupg
  • sudo install -m 0755 -d /etc/apt/keyrings

  • curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
    

  • sudo chmod a+r /etc/apt/keyrings/docker.gpg

  • echo \
      "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
      $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
      sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
    

  • sudo apt-get update
  • sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
• Add Current User to Docker Group
  • By default, only the root user can do docker commands, you need to add your current user to the docker group in order to allow it to run docker commands.
    • sudo groupadd docker
    • sudo gpasswd -a $USER docker
    • newgrp docker
• Test Docker installation
  • docker run hello-world
    • This command runs a test docker container to see if your installation functions normally. The output should look like the image below
    • 

• NOTES
  • If you already have nginx deployed, skip to the “Add a website for URL redirect” step.
•  Assumptions
  • You have an Ubuntu 22.04 Linux server created
    • 4GB of RAM
    • 30GB of Drive Storage
• Prerequisites
  • You have a DNS “A” record redirecting all subdomains to your machine.
    • For instance, if your domain is “example.com”, you need to create an A record of “*” to redirect everything to anysubdomain.example.com.
• Download openssh (OPTIONAL)
  • This allows the user to ssh into the machine to allow for easier copying and pasting of commands
    • sudo -i
      • This puts the current terminal as a super user (can run commands as an admin).
    • apt-get update
      • Updates the current list of Linux packages.
    • apt-get upgrade
      • Upgrades all of the packages/updates the actual software
    • apt-get install openssh-server
      • This installs openssh so you can ssh to the Linux machine
  • Connect to the Linux machine via SSH (use putty or the cmd)
• Install nginx
  • apt install nginx
    • This installs the nginx software and engine
  • systemctl status nginx
    • this will check the status of nginx, ensuring the it is running
• Add a website for URL redirect
  • cd /etc/nginx/sites-available
  • nano yourwebsite.example.com
    • You will want this to actually be the URL you are redirecting. For instance, test.nationaltrail.us. “test” is the subdomain and “nationaltrail.us” is the domain we own.
  • Add the following configuration to the file
    • server {
      server_name test.nationaltrail.us;location / {
      proxy_pass http://10.10.8.112; # Point to test web server
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      }
      }
      • This redirects “test.nationaltrail.us” to “http://10.10.8.112”
  • Link sites-available to sites-enabled
    • ln -s /etc/nginx/sites-available/test.nationaltrail.us /etc/nginx/sites-enabled
      • This creates a link from sites-available to sites-enabled to allow nginx to start redirecting/have access to the configuration
  • Test the configuration
    • nginx -t
      • This should have an output such as “the configuration is OK”
  • Restart the nginx service
    • systemctl reload nginx
  • You should now ab able to go to “test.nationaltrail.us” to go to the website located at “10.10.8.112”
• Add Certbot and secure the site
  • At this point, you are only redirecting traffic, not adding an SSL certificate. This will add encryption to secure the site.
  • Install certbot and the nginx plugin
    • apt install certbot python3-certbot-nginx
  • Secure your website
    • certbot –nginx -d test.nationaltrail.us
  • Go through the installation process. This includes adding a notification email for certificate expiration AND choosing the option for REDIRECTING traffic to HTTPS. This is the second option if you are prompted for it.
  • Restart nginx
    • systemctl reload nginx
• Connect to your website! It should be secured unless the web app requires further proxy configuration (refer to the webpages documentation for this possibility).

• NOTE: You can do other commands such as copy files from the container to your VM, install tools in the container, and anything that can be done on a regular Ubuntu install. For more information, look in the DOCKER DOCUMENTATION.
•  Assumptions
  • You are using Ubuntu/Debian-based OS
  • You have Docker installed
  • You have a running container
  • You have access to the docker environment
• Get the docker ID and list the containers
  • Open Ubuntu Terminal
    • docker ps
      • This allows you to get a list of all docker containers
  • The docker ID is the first string of numbers and letters before your container
    • 
  • Enter the docker container
    • docker exec -it <your container ID> bash
      • docker exec -it 175cb158268e bash
• You can now execute commands in the container just as you could in a regular Ubuntu Server.

Here at school, and at my home, we use an SSD for the OS and programs, and a traditional HDD for user files and data.  This requires a registry edit before a user logs on, and has worked great to save SSD space for those items that need SSD speed.

However, it appears to cause issues with major updates in Windows.  While minor updates happen frequently, major ones always fail.  I needed the major upgrade to 21H2 to make Starfall work, so it was a priority!  🙂

To fix this issue I had to do the following.

1.  Logon with the admin account that was originally setup with windows.  This account is stored in the User folder where windows was installed.
2. Undo the registry hack to put user folders on the D: Drive.
   1. [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList] should be returned to %SystemDrive%\Users
3. Run the following in an administrator command prompt.  In short, it repairs the Windows installation folder.
   1. DISM.exe /Online /Cleanup-image /Restorehealth
4. Now I ran an upgrade from a flash drive with the current Windows release…and it worked!

Hope this helps anyone else with Windows upgrade issues, or more importantly, Starfall issues!

• Assumptions
  • You already have a vCenter virtual environment with permission to create virtual machines.
    • This environment is fully licensed.
  • You have prior knowledge of how to create a virtual machine (Can make a Windows Server 2022 machine).
• Prerequisites
  • On Personal Machine
    • Download and install the Tanzu Grid CLI tools from the VMWare website
      • DOWNLOAD LINK HERE
      • 
    • Download, install, and open Docker Desktop on your machine
      • DOWNLOAD LINK
      • 
  • On vCenter
    • You need to download the Tanzu base image (OVA file) from the VMware website, upload it to your server and convert it to a template. (When you do the initial setup on the Tanzu you will know what version you need)
      • DOWNLOAD LINK HERE
• Create an SSH Public key on your vCenter environment
  • Generate a public key using Puttygen or other key-generation softwares
  • SSH to the vCenter server using Putty or another SSH service
  • Enter the following commands and change the following lines
    • vi /etc/ssh/sshd_config
      • CHANGE: PermitRootLogin no” to “PermitRootLogin yes”
    • vi /etc/ssh/keys-root/authorized_keys
      • Add the key you created
    • /etc/init.d/SSH restart
• Do the Tanzu Installation
  • Enter the following commands into the command line
    • tanzu init
    • tanzu mc create -u
  • You will be presented with a web GUI for the Tanzu installation
    • Select “Deploy” under the VMware vSphere option
    • 
  • Under the IaaS Provider Option
    • IP or FQDN
      • Enter the IP address/DNS name of your virtual environment
    • Username
      • Enter the username of an administrator account for vCenter
    • Password
      • The password to the account name you provided
    • Disable verification
      • Unchecked
    • SSL Thumbprint Verification
      • Check “Disable Verification”
    • Click Connect
    • Select the datacenter you have set up in vCenter
    • SSH Public Key
      • Enter the same public key you put in your vCenter config files
  • Under Management Cluster Settings Tab
    • Deployment 
      • Select the desired instance type; I used small
    • Management Cluster Name 
      • Give your new cluster a name
    • Control Plane Endpoint Provider
      • Kube-vip (for simplicity’s sake)
    • Worker Node Instance Type
      • I used small
    • Control Plane Endpoint
      • Give your cluster an unused IP address
  • VMware NSX Advanced Load Balancer
    • LEAVE THIS BLANK
  • Metadata
    • LEAVE THIS BLANK
  • Resources
    • VM Folder
      • Select the folder that you want your VMs to be a part of.
    • Datastore
      • Select the datastore you want your machines to be stored
  • Kubernetes Network
    • Network Name
      • Select the port group/vCenter network you want to use.
    • Cluster Service CIDR
      • Leave Default 
        • CLUSTER SERVICE CIDR
          • 100.64.0.0/13
        • CLUSTER POD CIDR
          • 100.96.0.0/11
  • Identity Management
    • DISABLE THE CHECKBOX TO BLANK THIS OUT
  • OS Image
    • Select the template you made earlier in the installation process
  • Select “Review Configuration” and finish the installation
• Deploy an Azure DevOps machine
  • Create a Windows Server VM (I used 2022)
    • Download the Azure DevOps Server Tools
      • DOWNLOAD LINK HERE
    • This is a very typical setup wizard (basically just clicking next 5 times). It will install Java and SQL for you if you do not have them already.
  • After an uneasy amount of time, the installation will be complete (you may need to restart). You can then open the Azure DevOps Server Administration Console.
    • You can then connect to the web GUI through a web browser
      • http://azure-devops OR http://SERVER _IP_ADDRESS
  • The username and password are the same as the Admin user account on the Windows Server 2022 server
    • You can add other users in the administration console under the “Application Tire” tab under “Administration Console Users”
• Connecting Everything Together – The Finale
  • In the Azure-DevOps web GUI, go to “Collection Settings” located at the bottom left.
  • Go to the “Agent Pools” tab toward the bottom left
  • Select the Default Agent pool
    • Select “New Agent” at the top right
    • Follow the instructions prompted to you on the screen (downloading a zip file, making a few directories, and such).
    • I HAVE USED THE SAME 2022 SERVER FOR BOTH HOSTING THE DEVOPS WEBSITE AND THE AGENT TOOLS.
      • Run “config.cmd”
        • Enter Server URL
          • https://AZURE_DEVOPS_IP_ADDRESS
        • Press enter for the integrated authentication type
        • Run this as a service so it starts when the machine starts
  • Go to the “Agents” tab in the Default Agent Pool section
    • 
  • There should now be an Agent that is currently online
  • Connect the Kubernetes cluster to a pipeline
    • In the Azure DevOps console, create a new Project on the main screen
      • Go to the “pipelines” selection on the left
      • Select “Environments”
      • Create Environment
        • Give it a name
        • Select Kubernetes as the Resource
          • Provider 
            • Leave Default
          • Cluster Name
            • This will be what you named your cluster when you created the Tanzu Cluster
          • Namespace
            • You can use the “default” namespace for simplicity’s sake
          • Server URL
            • https://YOUR_CLUSTER_IP_ADDRESS_YOU_ASSIGNED
          • Secret
            • In a command line interface, connect to your Tanzu Cluster
              • tanzu init
              • tanzu login
                • Select your cluster and hit enter
              • Create a new service account
                • Download the file located HERE 
                • Change your directory to your downloads folder in the command line
                • Execute a command to run the YAML file
                • kubectl apply -f azure-devops-service-account.yaml
            • Get the token for the service account
              • kubectl get serviceAccounts <service-account-name> -n <namespace> -o=jsonpath={.secrets[*].name}
            • Use the token to get the secret
              • kubectl get secret <service-account-secret-name> -n <namespace> -o json
                • Replace the <Service Account Secret Name> with the token you got from the previous command
            • Enter the output you got from that in the Azure DevOps “Secret” textbox.
          • Check the “Accept untrusted certificates” checkbox
          • Finish by clicking “Validate and Create”