Search
Subclassing pathlib.PosixPath broken since Python 3.12 (actually its fixed, but workaround broken)
Solved: Thanks to a user with this reply: https://programming.dev/comment/10034690
---
cross-posted from: https://beehaw.org/post/13901165
> Hi all. I have a little problem and don't know how to solve. A CLI program in Python is broken since Python 3.12. It was working in Python 3.11. The reason is, that Python 3.12 changed how subclassing of a pathlib.Path works (basically fixed an issue), which now breaks a workaround.
>
> #### The class in question is:
>
> python > class File(PosixPath): > def __new__(cls, *args: Any, **kwargs: Any) -> Any: > return cls._from_parts(args).expanduser().resolve() # type: ignore > > def __init__(self, source: str | Path, *args: Any) -> None: > super().__init__() > self.__source = Path(source) > > @property > def source(self) -> Path: > return self.__source > > @property > def modified(self) -> Time: > return Time.fromtimestamp(os.path.getmtime(self)) > > @property > def changed(self) -> Time: > return Time.fromtimestamp(os.path.getctime(self)) > > @property > def accessed(self) -> Time: > return Time.fromtimestamp(os.path.getatime(self)) > > # Calculate sha512 hash of self file and compare result to the > # checksum found in given file. Return True if identical. > def verify_sha512(self, file: File, buffer_size: int = 4096) -> bool: > compare_hash: str = file.read_text().split(" ")[0] > self_hash: str = "" > self_checksum = hashlib.sha512() > with open(self.as_posix(), "rb") as f: > for chunk in iter(lambda: f.read(buffer_size), b""): > self_checksum.update(chunk) > self_hash = self_checksum.hexdigest() > return self_hash == compare_hash >
>
> #### and I get this error when running the script:
>
> > Traceback (most recent call last): > File "/home/tuncay/.local/bin/geprotondl", line 1415, in > sys.exit(main()) > ^^^^^^ > File "/home/tuncay/.local/bin/geprotondl", line 1334, in main > arguments, status = parse_arguments(argv) > ^^^^^^^^^^^^^^^^^^^^^ > File "/home/tuncay/.local/bin/geprotondl", line 1131, in parse_arguments > default, status = default_install_dir() > ^^^^^^^^^^^^^^^^^^^^^ > File "/home/tuncay/.local/bin/geprotondl", line 1101, in default_install_dir > steam_root: File = File(path) > ^^^^^^^^^^ > File "/home/tuncay/.local/bin/geprotondl", line 97, in __new__ > return cls._from_parts(args).expanduser().resolve() # type: ignore > ^^^^^^^^^^^^^^^ > AttributeError: type object 'File' has no attribute '_from_parts'. Did you mean: '_load_parts'? >
>
> #### Now replacing _from_parts
with _load_parts
does not work either and I get this message in that case:
>
> > Traceback (most recent call last): > File "/home/tuncay/.local/bin/geprotondl", line 1415, in > sys.exit(main()) > ^^^^^^ > File "/home/tuncay/.local/bin/geprotondl", line 1334, in main > arguments, status = parse_arguments(argv) > ^^^^^^^^^^^^^^^^^^^^^ > File "/home/tuncay/.local/bin/geprotondl", line 1131, in parse_arguments > default, status = default_install_dir() > ^^^^^^^^^^^^^^^^^^^^^ > File "/home/tuncay/.local/bin/geprotondl", line 1101, in default_install_dir > steam_root: File = File(path) > ^^^^^^^^^^ > File "/home/tuncay/.local/bin/geprotondl", line 97, in __new__ > return cls._load_parts(args).expanduser().resolve() # type: ignore > ^^^^^^^^^^^^^^^^^^^^^ > File "/usr/lib/python3.12/pathlib.py", line 408, in _load_parts > paths = self._raw_paths > ^^^^^^^^^^^^^^^ > AttributeError: 'tuple' object has no attribute '_raw_paths' >
>
> #### I have searched the web and don't understand how to fix this. Has anyone an idea what to do?
Why won't my Python unit tests run?
Hello! I'm attempting to follow some tutorials on unit testing with Python. One of them is a video tutorial Unit Tests in Python on the Socratica channel. Everyone in the comments seems to be making out just fine, and I’m following the instructor’s directions to the letter, yet I get a different result. It’s driving me mad lol.
In the video, the instructor creates two text files, one called circles.py
in which she defines a function circle_area(r)
, and another called test_circles.py
in which she writes some unit tests. In my attempt to follow along, I've ended up with two files structured like so:
/home/yo_scottie_oh/Projects/PythonTutorials/Socratica/Circles ├── circles.py └── test_circles.py
circles.py
:
``` from math import pi
def circle_area(r): return pi*(r**2)
Test function
radii = [2, 0, -3, 2 + 5j, True, "radius"] message = "Area of circles with r = {radius} is {area}."
for r in radii: A = circle_area(r) print(message.format(radius=r,area=A)) ```
test_circles.py
:
``` import unittest from circles import circle_area from math import pi
class TestCircleArea(unittest.TestCase): def test_area(self): # Test areas when radius >=0 self.assertAlmostEqual(circle_area(1),pi) self.assertAlmostEqual(circle_area(0),0) self.assertAlmostEqual(circle_area(2.1),pi*2.1**2) ```
Where I'm getting tripped up is at 4:32 in the video, the instructor says to run the unit tests by opening a shell, going to the directory that contains both the circles and test_circles modules, and issuing the following command: python -m unittest test_circles
.
Instructor's result (it runs the unit test):
``` Ran 1 test in 0.000s
OK ```
My result (it seems to execute circles.py
itself):
[yo_scottie_oh@nobara Circles]$ python -m unittest test_circles Area of circles with r = 2 is 12.566370614359172. Area of circles with r = 0 is 0.0. Area of circles with r = -3 is 28.274333882308138. Area of circles with r = (2+5j) is (-65.97344572538566+62.83185307179586j). Area of circles with r = True is 3.141592653589793. Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/usr/lib64/python3.11/unittest/__main__.py", line 18, in <module> main(module=None) File "/usr/lib64/python3.11/unittest/main.py", line 101, in __init__ self.parseArgs(argv) File "/usr/lib64/python3.11/unittest/main.py", line 150, in parseArgs self.createTests() File "/usr/lib64/python3.11/unittest/main.py", line 161, in createTests self.test = self.testLoader.loadTestsFromNames(self.testNames, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.11/unittest/loader.py", line 232, in loadTestsFromNames suites = [self.loadTestsFromName(name, module) for name in names] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.11/unittest/loader.py", line 232, in <listcomp> suites = [self.loadTestsFromName(name, module) for name in names] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.11/unittest/loader.py", line 162, in loadTestsFromName module = __import__(module_name) ^^^^^^^^^^^^^^^^^^^^^^^ File "/home/yo_scottie_oh/Projects/PythonTutorials/Socratica/Circles/test_circles.py", line 4, in <module> from circles import circle_area File "/home/yo_scottie_oh/Projects/PythonTutorials/Socratica/Circles/circles.py", line 14, in <module> A = circle_area(r) ^^^^^^^^^^^^^^ File "/home/yo_scottie_oh/Projects/PythonTutorials/Socratica/Circles/circles.py", line 6, in circle_area return pi*(r**2) ~^^~ TypeError: unsupported operand type(s) for ** or pow(): 'str' and 'int' [yo_scottie_oh@nobara Circles]$
I've been banging my head against the wall for hours now trying to figure out why when I execute the same command as the instructor, it appears to execute my Python scripts themselves instead of running the unit tests.
Other things I've tried:
I've read the Python documentation on unit testing. I tried adding this to the end of the test_circles.py document, but that did not change anything.
if __name__ == '__main__': unittest.main()
I've tried following this other written tutorial. After I create the text documents and organize them in the separate shapes and tests folders and run the command python -m unittest discover -v
, again I get a different result from the author.
Author's result:
``` test_area (test_circle.TestCircle) ... ok test_circle_instance_of_shape (test_circle.TestCircle) ... ok test_create_circle_negative_radius (test_circle.TestCircle) ... ok test_area (test_square.TestSquare) ... ok test_create_square_negative_length (test_square.TestSquare) ... ok test_square_instance_of_shape (test_square.TestSquare) ... ok
---------------------------------------------------------------------- Ran 6 tests in 0.002s
OK ```
My result:
``` [yo_scottie_oh@nobara test]$ python -m unittest discover -v test_circle (unittest.loader._FailedTest.test_circle) ... ERROR test_square (unittest.loader._FailedTest.test_square) ... ERROR
====================================================================== ERROR: test_circle (unittest.loader._FailedTest.test_circle) ---------------------------------------------------------------------- ImportError: Failed to import test module: test_circle Traceback (most recent call last): File "/usr/lib64/python3.11/unittest/loader.py", line 419, in _find_test_path module = self._get_module_from_name(name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.11/unittest/loader.py", line 362, in _get_module_from_name import(name) File "/home/yo_scottie_oh/Projects/PythonTutorials/PythonUnitTesting/test/test_circle.py", line 4, in <module> from shapes.circle import Circle ModuleNotFoundError: No module named 'shapes'
====================================================================== ERROR: test_square (unittest.loader._FailedTest.test_square) ---------------------------------------------------------------------- ImportError: Failed to import test module: test_square Traceback (most recent call last): File "/usr/lib64/python3.11/unittest/loader.py", line 419, in _find_test_path module = self._get_module_from_name(name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.11/unittest/loader.py", line 362, in _get_module_from_name import(name) File "/home/yo_scottie_oh/Projects/PythonTutorials/PythonUnitTesting/test/test_square.py", line 3, in <module> from shapes.square import Square ModuleNotFoundError: No module named 'shapes'
---------------------------------------------------------------------- Ran 2 tests in 0.000s
FAILED (errors=2) ```
So yeah… this brings me to my question: What’s the obvious thing that everybody else gets that I'm missing? Is the tutorial outdated? Is it because the instructor is on Windows and I’m on Linux? Why won’t my unit tests run?
How to Retrieve All Posts for a User or Community on Lemmy Using Pythörhead Library?
I have encountered a situation where I need to retrieve all posts for a user or community on Lemmy and store them in a database. Due to a recent mishap, I had to revert to an older version of the database, resulting in a loss of some posts.
To overcome this issue, I am seeking guidance on how to efficiently retrieve all posts for a user or community on Lemmy and extract important information such as the URL, Title, Body, and Posted Timestamp.
To accomplish this task, I am utilizing the Pythörhead library, which provides a Python interface for interacting with the Lemmy API.
One approach I have considered is searching for each URL before posting to avoid duplicating posts. However, this method would be very slow and would create a significant load on the instance, potentially impacting performance.
Any suggestions on how best to solve this issue would be invaluable.
For reference, this is the Lemmy API endpoint that can be utilized for retrieving posts is.
Thank you in advance for your assistance!
How would you debug this script without creating many posts?
I won't know if this script works until I run it and see the errors but the comments won't start to generate until after all the posts so I can't debug that part until I've already created too much content.
```python import sqlite3 import requests from pythorhead import Lemmy import schedule import time import logging from config import *
logging.basicConfig( level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s %(message)s", handlers=[logging.FileHandler("debug.log"), logging.StreamHandler()], )
def initialize_database(): conn = sqlite3.connect(DB_FILE) cursor = conn.cursor() cursor.execute(""" CREATE TABLE IF NOT EXISTS posts ( github_url TEXT PRIMARY KEY, lemmy_post_id INTEGER, lemmy_post_name TEXT, lemmy_post_body TEXT ) """) cursor.execute(""" CREATE TABLE IF NOT EXISTS comments ( github_comment_id INTEGER PRIMARY KEY, lemmy_comment_id INTEGER, comment_user TEXT, comment_body TEXT ) """) conn.commit() return conn
def initialize_lemmy_instance(): lemmy = Lemmy(LEMMY_INSTANCE_URL) lemmy.log_in(LEMMY_USERNAME, LEMMY_PASSWORD) logging.info("Initialized Lemmy instance") return lemmy
def discover_community(lemmy, community_name): community_id = lemmy.discover_community(community_name) logging.info(f"Discovered community {community_name} with ID {community_id}") return community_id
def fetch_github_issues(repo): url = f"{GITHUB_API_BASE}/repos/{repo}/issues" headers = {"Accept": "application/vnd.github+json"} response = requests.get(url, headers=headers) logging.info(f"Fetched issues from {url}") return response.json()
def extract_issue_info(issue, repo): issue_url = issue["html_url"] issue_state = "[Closed]" if issue["state"] == "closed" else "" repo_abbr = "[BE]" if "lemmy" in repo else "[UI]" issue_title = f"{issue_state}{repo_abbr} {issue['title']} #{issue['number']}" issue_body = issue["body"] return issue_url, issue_title, issue_body
def post_issues_to_lemmy(lemmy, community_id, repo): conn = sqlite3.connect(DB_FILE) cursor = conn.cursor()
issues = fetch_github_issues(repo) for issue in issues: issue_url, issue_title, issue_body = extract_issue_info(issue, repo)
cursor.execute("SELECT lemmy_post_id FROM posts WHERE github_url=?", (issue_url,)) existing_post = cursor.fetchone()
if not existing_post: post = lemmy.post.create(community_id, issue_title, url=issue_url, body=issue_body)["post_view"]["post"] lemmy_post_id = post["id"] lemmy_post_name = post["name"] lemmy_post_body = post["body"] cursor.execute("INSERT INTO posts (github_url, lemmy_post_id, lemmy_post_name, lemmy_post_body) VALUES (?, ?, ?, ?)", (issue_url, lemmy_post_id, lemmy_post_name, lemmy_post_body)) conn.commit() logging.info(f"Posted issue {issue_title} to community {community_id}")
def fetch_github_comments(repo, issue_number): url = f"{GITHUB_API_BASE}/repos/{repo}/issues/{issue_number}/comments" headers = {"Accept": "application/vnd.github+json"} response = requests.get(url, headers=headers) logging.info(f"Fetched comments for issue #{issue_number}") return response.json()
def post_comments_to_lemmy(lemmy, post_id, repo, issue_number): conn = sqlite3.connect(DB_FILE) cursor = conn.cursor()
github_comments = fetch_github_comments(repo, issue_number) for comment in github_comments: github_comment_id = comment["id"] cursor.execute("SELECT lemmy_comment_id FROM comments WHERE github_comment_id=?", (github_comment_id,)) existing_comment = cursor.fetchone()
if not existing_comment: comment_user = comment["user"]["login"] comment_body = comment["body"] lemmy_comment_id = lemmy.comment.create(post_id, comment_body)["comment"]["id"]
cursor.execute("INSERT INTO comments (github_comment_id, lemmy_comment_id, comment_user, comment_body) VALUES (?, ?, ?, ?)", (github_comment_id, lemmy_comment_id, comment_user, comment_body)) conn.commit() logging.info(f"Posted comment {github_comment_id} to lemmy post {post_id}")
Fetch the GitHub issue number and Lemmy post ID for each issue
def fetch_issue_data(repo): conn = sqlite3.connect(DB_FILE) cursor = conn.cursor() cursor.execute("SELECT github_url, lemmy_post_id FROM posts WHERE github_url LIKE ?", (f"https://github.com/{repo}/issues/%",)) issue_data = cursor.fetchall() return issue_data
def extract_issue_number(github_url): return int(github_url.split("/")[-1])
def main(): logging.info("Running main function") initialize_database() lemmy = initialize_lemmy_instance() community_id = discover_community(lemmy, LEMMY_COMMUNITY_NAME) for repo in REPOSITORIES: post_issues_to_lemmy(lemmy, community_id, repo) issue_data = fetch_issue_data(repo) for github_url, lemmy_post_id in issue_data: issue_number = extract_issue_number post_comments_to_lemmy(lemmy, lemmy_post_id, repo, issue_number)
def run_periodically(): main() schedule.every(2).hours.do(main)
while True: schedule.run_pending() time.sleep(60)
if name == "main": logging.info("Starting script") run_periodically() ```
Help with spotipy
I am trying to create a playlist with spotify and the spotipy library in python. However, I keep getting a "No token provided" error when making my API request. However, if I use the same token with a curl request, it works! Can someone please help. This is my code:
``` auth_manager = SpotifyOAuth(client_id=CLIENT, client_secret=SECRET, redirect_uri="http://example.com/", scope=SCOPE, username=spotify_display_name ) token = auth_manager.get_access_token( as_dict=False, check_cache=True )
sp = spotipy.Spotify(auth_manager=auth_manager, auth=token ) user_dict = sp.current_user() user_id = user_dict["id"] print(f"Welcome, {user_dict['display_name']}")
SEARCH
QUERY FORMAT: "track: track-name year: YYYY"
spotify_search_endpoint = "https://api.spotify.com/v1/search/" test_query = "track:Hangin'+Tough year:1989"
search_parameters = { "q": format_query(test_query), "type": "track" }
results = sp.search(q=search_parameters["q"]) print(results) ```
output:
{'tracks': {'href': 'https://api.spotify.com/v1/search?query=track%3AHangin%27%2BTough%2520year%3A1989&type=track&offset=0&limit=10', 'items': [], 'limit': 10, 'next': None, 'offset': 0, 'previous': None, 'total': 0}}
{ "error": { "status": 401, "message": "No token provided" } }
This is really frustrating! The authentication is working, otherwise the token wouldn't have been valid for the curl request. I must be doing something wrong with spotipy.