Commit Graph

27 Commits

Author SHA1 Message Date
Driss Guessous dcafe3f271 Updates to the release notes scripts and documentation (#94560)
# Summary
This PR made some significant changes to the scripts around Release Scripts. At a high level:
- Turned the quips into docs and updated links
- Update the common.categorizes list in the hopes to make this the source of truth for releases- This is hard since the release_notes labels can be changed at will. An alternative would be to poll from github api. However, I think that is overkill. The notebook does a set compare and will show you knew categories. I think we want this to be manual so that the release note engineer will decided how to categorize.
- Create cateogry group from speaking with folks on distributed and AO that told me these different release categories can be merged.
- I am the newest person to Core and don't use ghstack soo made token getting a lil more generic.
- Added a classifier.py file. This file will train a commit categorizer for you, hopefully with decent accuracy. I was able to achieve 75% accuracy. I drop the highest frequency class - "skip" since this creates a more useful cateogrizer.
- I updated the categorize.py script so that the prompt will be what the classifier thinks, gated by a flag.
- Added a readme that will hopefully help future release notes engineers.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/94560
Approved by: https://github.com/albanD
2023-03-16 00:09:26 +00:00
Aaron Gokaslan dd5e6e8553 [BE]: Merge startswith calls - rule PIE810 (#96754)
Merges startswith, endswith calls to into a single call that feeds in a tuple. Not only are these calls more readable, but it will be more efficient as it iterates through each string only once.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/96754
Approved by: https://github.com/ezyang
2023-03-14 22:05:20 +00:00
Aaron Gokaslan 67d9790985 [BE] Apply almost all remaining flake8-comprehension checks (#94676)
Applies the remaining flake8-comprehension fixes and checks. This changes replace all remaining unnecessary generator expressions with list/dict/set comprehensions which are more succinct, performant, and better supported by our torch.jit compiler. It also removes useless generators such as 'set(a for a in b)`, resolving it into just the set call.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/94676
Approved by: https://github.com/ezyang
2023-02-12 01:01:25 +00:00
Mikayla Gawarecki df13247e67 small bugfixes to release notes script (#94536)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/94536
Approved by: https://github.com/drisspg
2023-02-10 01:23:07 +00:00
Xuehai Pan a229b4526f [BE] Prefer dash over underscore in command-line options (#94505)
Preferring dash over underscore in command-line options. Add `--command-arg-name` to the argument parser. The old arguments with underscores `--command_arg_name` are kept for backward compatibility.

Both dashes and underscores are used in the PyTorch codebase. Some argument parsers only have dashes or only have underscores in arguments. For example, the `torchrun` utility for distributed training only accepts underscore arguments (e.g., `--master_port`). The dashes are more common in other command-line tools. And it looks to be the default choice in the Python standard library:

`argparse.BooleanOptionalAction`: https://github.com/python/cpython/blob/4a9dff0e5adc91cbb1ed68c495dac64ccfe608bd/Lib/argparse.py#L893-L895

```python
class BooleanOptionalAction(Action):
    def __init__(...):
            if option_string.startswith('--'):
                option_string = '--no-' + option_string[2:]
                _option_strings.append(option_string)
```

It adds `--no-argname`, not `--no_argname`. Also typing `_` need to press the shift or the caps-lock key than `-`.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/94505
Approved by: https://github.com/ezyang, https://github.com/seemethere
2023-02-09 20:16:49 +00:00
Jane Xu 0ecb071fc4 [BE][CI] change references from .jenkins to .ci (#92624)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/92624
Approved by: https://github.com/ZainRizvi, https://github.com/huydhn
2023-01-30 22:50:07 +00:00
David Berard c413a32135 Release note script: match topics with spaces or underscores (#87011)
e.g. match "new features" in the category as "new_features"
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87011
Approved by: https://github.com/albanD, https://github.com/soulitzer
2022-10-19 02:28:45 +00:00
John Clow 3c4c7d3e6b [Release Notes] fix bug with categorize call (#81284)
This was pointed out by @kit1980 in #78190
Pull Request resolved: https://github.com/pytorch/pytorch/pull/81284
Approved by: https://github.com/kit1980
2022-07-12 19:02:15 +00:00
John Clow 7fd0cf5581 [Release Notes] Add way to export result from Google Sheets to Markdown (#79911)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79911
Approved by: https://github.com/soulitzer
2022-07-07 22:42:46 +00:00
John Clow 6d4410b8c6 [Release Notes] Simple script to merge categories (#79910)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79910
Approved by: https://github.com/soulitzer
2022-07-07 22:42:46 +00:00
John Clow 755861063d Adding additional topics to align with github topics list (#79909)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79909
Approved by: https://github.com/soulitzer
2022-07-07 22:42:46 +00:00
John Clow 770fc74e33 [Release Notes] Add Github PR link to csv export (#79908)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79908
Approved by: https://github.com/soulitzer
2022-07-07 22:42:46 +00:00
John Clow ad6328ea51 [Release Notes] Adding CSV Category Export (#78212)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78212
Approved by: https://github.com/soulitzer
2022-07-07 22:42:46 +00:00
John Clow 62bf807113 Always use the CommitCache, and make it a singleton (#78203)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78203
Approved by: https://github.com/soulitzer
2022-07-07 22:42:46 +00:00
John Clow da549f58d5 Adding Author and Accepters information into pytorch release notes gen (#78190)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78190
Approved by: https://github.com/soulitzer, https://github.com/malfet
2022-07-07 22:42:46 +00:00
John Clow 8549fafd36 Refactoring release not script to use dataclasses and have a shorter test. (#78189)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78189
Approved by: https://github.com/soulitzer
2022-07-07 22:42:46 +00:00
Brian Hirsh 43f6d79e51 update release notes script to automatically grab labels from the PR
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75020

Approved by: https://github.com/albanD, https://github.com/anjali411
2022-05-12 18:39:24 +00:00
Brian Hirsh 5ed7312081 release notes script changes
Pull Request resolved: https://github.com/pytorch/pytorch/pull/72154

Approved by: https://github.com/albanD, https://github.com/anjali411
2022-05-12 18:39:24 +00:00
anjali411 211deb0364 Fix CI quick-checks (#71773)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/71773

Test Plan: Imported from OSS

Reviewed By: pbelevich

Differential Revision: D33770042

Pulled By: anjali411

fbshipit-source-id: 9dd3f8c8592663d385ab0cd4376aaa4b9c7d9ec2
(cherry picked from commit 739c8885c7)
2022-01-25 18:32:13 +00:00
anjali411 d32b7d9585 Logic to auto-categorize commits (#64929)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/64929

Auto categorized 63% of the commits for PyTorch 1.10 release (2.2k out of 3.4k commits)

Test Plan: Imported from OSS

Reviewed By: malfet

Differential Revision: D33768760

Pulled By: anjali411

fbshipit-source-id: 0655090af83e923f8c26fa1ce9f190edc542b97e
(cherry picked from commit 2fe30f77b8)
2022-01-25 17:32:41 +00:00
Nikita Shulga add49e7e4e Enforce PEP263 for PyTorch python codebase (#55346)
Summary:
All python files containing non-ASCII characters should be correctly annotated with `# -*- coding: utf-8 -*-` comment

Delete number of superfluous UTF-8 characters, most commonly UTF-8 opening closing quotation mark U+2019 (’) instead of ascii apostrophe ', for example `Module’s`->`Module's`

Pull Request resolved: https://github.com/pytorch/pytorch/pull/55346

Reviewed By: samestep

Differential Revision: D27582044

Pulled By: malfet

fbshipit-source-id: c1cd89655915858ff3a41f675cdfffff795a8e44
2021-04-06 18:31:38 -07:00
Alban Desmaison f83668b4e5 Update release notes scripts following runbook update (#54594)
Summary:
This adds:
- new categories
- global commit counter
- support for new "Reverted" label on PRs
- new export system to multiple files

Pull Request resolved: https://github.com/pytorch/pytorch/pull/54594

Reviewed By: H-Huang

Differential Revision: D27396011

Pulled By: albanD

fbshipit-source-id: ca1ec3a1b90221ba26fd8b053dfb10f614f05909
2021-04-01 07:55:16 -07:00
Sam Estep 5bcbbf5373 Lint trailing newlines (#54737)
Summary:
*Context:* https://github.com/pytorch/pytorch/issues/53406 added a lint for trailing whitespace at the ends of lines. However, in order to pass FB-internal lints, that PR also had to normalize the trailing newlines in four of the files it touched. This PR adds an OSS lint to normalize trailing newlines.

The changes to the following files (made in 54847d0adb) are the only manually-written parts of this PR:

- `.github/workflows/lint.yml`
- `mypy-strict.ini`
- `tools/README.md`
- `tools/test/test_trailing_newlines.py`
- `tools/trailing_newlines.py`

I would have liked to make this just a shell one-liner like the other three similar lints, but nothing I could find quite fit the bill. Specifically, all the answers I tried from the following Stack Overflow questions were far too slow (at least a minute and a half to run on this entire repository):

- [How to detect file ends in newline?](https://stackoverflow.com/q/38746)
- [How do I find files that do not end with a newline/linefeed?](https://stackoverflow.com/q/4631068)
- [How to list all files in the Git index without newline at end of file](https://stackoverflow.com/q/27624800)
- [Linux - check if there is an empty line at the end of a file [duplicate]](https://stackoverflow.com/q/34943632)
- [git ensure newline at end of each file](https://stackoverflow.com/q/57770972)

To avoid giving false positives during the few days after this PR is merged, we should probably only merge it after https://github.com/pytorch/pytorch/issues/54967.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/54737

Test Plan:
Running the shell script from the "Ensure correct trailing newlines" step in the `quick-checks` job of `.github/workflows/lint.yml` should print no output and exit in a fraction of a second with a status of 0. That was not the case prior to this PR, as shown by this failing GHA workflow run on an earlier draft of this PR:

- https://github.com/pytorch/pytorch/runs/2197446987?check_suite_focus=true

In contrast, this run (after correcting the trailing newlines in this PR) succeeded:

- https://github.com/pytorch/pytorch/pull/54737/checks?check_run_id=2197553241

To unit-test `tools/trailing_newlines.py` itself (this is run as part of our "Test tools" GitHub Actions workflow):
```
python tools/test/test_trailing_newlines.py
```

Reviewed By: malfet

Differential Revision: D27409736

Pulled By: samestep

fbshipit-source-id: 46f565227046b39f68349bbd5633105b2d2e9b19
2021-03-30 13:09:52 -07:00
albanD e8ee35a666 Add script to compare namespace content for release cleanup (#51685)
Summary:
Usage explanation will be in the release note runbook.

This allows to generate diffs like:
```
Processing torch.nn
Things that were added:
{'quantizable', 'ChannelShuffle', 'LazyConvTranspose2d', 'LazyConv2d', 'LazyConvTranspose3d', 'LazyConv1d', 'GaussianNLLLoss', 'LazyConv3d', 'PixelUnshuffle', 'UninitializedParameter', 'LazyLinear', 'LazyConvTranspose1d'}

Things that were removed:
set()
```

This can then be shared with module owners along with the commits to help them validate that the namespace changes for their submodule is as expected.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/51685

Reviewed By: zhangguanheng66

Differential Revision: D26260258

Pulled By: albanD

fbshipit-source-id: 40e40f86314e17246899d01ffa4b2631e93b52f7
2021-02-05 07:54:00 -08:00
Thomas Zhang d78b638a31 Convert string => raw strings so char classes can be represented in Python regex (#50239)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/50239

Convert regex strings that have character classes (e.g. \d, \s, \w, \b, etc) into raw strings so they won't be interpreted as escape characters.

References:
Python RegEx - https://www.w3schools.com/python/python_regex.asp
Python Escape Chars - https://www.w3schools.com/python/gloss_python_escape_characters.asp
Python Raw String - https://www.journaldev.com/23598/python-raw-string
Python RegEx Docs - https://docs.python.org/3/library/re.html
Python String Tester - https://www.w3schools.com/python/trypython.asp?filename=demo_string_escape
Python Regex Tester - https://regex101.com/

Test Plan: To find occurrences of regex strings with the above issue in VS Code, search using the regex \bre\.[a-z]+\(['"], and under 'files to include', use /data/users/your_username/fbsource/fbcode/caffe2.

Reviewed By: r-barnes

Differential Revision: D25813302

fbshipit-source-id: df9e23c0a84c49175eaef399ca6d091bfbeed936
2021-01-08 11:17:17 -08:00
Richard Barnes 5acb1cc1df Drop unused imports from scripts (#49956)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/49956

From
```
./python/libcst/libcst codemod remove_unused_imports.RemoveUnusedImportsWithGlean --no-format caffe2/
```

Test Plan: Standard sandcastle tests

Reviewed By: xush6528

Differential Revision: D25727347

fbshipit-source-id: 74d0a08aa0cfd0f492688a2b8278a0c65fd1deba
2021-01-04 16:08:28 -08:00
Alban Desmaison 68954fe897 Add release note scripts (#47360)
Summary:
First commit contains the initial code from Richard's branch.
Second commit are the changes that I made during the writing process
Third commit is the update to support category/topic pair for each commit

Pull Request resolved: https://github.com/pytorch/pytorch/pull/47360

Reviewed By: ejguan

Differential Revision: D24741003

Pulled By: albanD

fbshipit-source-id: d0fcc6765968dc1732d8a515688d11372c7e653d
2020-11-05 06:43:24 -08:00